2023-12-16T21:27:28.161 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-16T21:27:28.161 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --verbose --name yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi --description fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} --archive /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866 -- /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/orig.config.yaml 2023-12-16T21:27:28.205 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T21:27:28.293 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866 branch: wip-yuri8-testing-2023-12-15-0911 description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} email: yweinste@redhat.com first_in_suite: false job_id: '7494866' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri8-testing-2023-12-15-0911 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: c7586e95519b7be76f3b6e3a4b333888baf46bc9 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c7586e95519b7be76f3b6e3a4b333888baf46bc9 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri8-testing-2023-12-15-0911 sha1: c7586e95519b7be76f3b6e3a4b333888baf46bc9 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 2960 sha1: c7586e95519b7be76f3b6e3a4b333888baf46bc9 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri8-testing-2023-12-15-0911 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: c7586e95519b7be76f3b6e3a4b333888baf46bc9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwIPeHm8uOK67B7OzSC8aqMeMUOaYZkPomxzmiewZk1jLW5r1rcwjDi/6KEnxjGGz0NDRq/V4VbNnS8qtp6GvU= smithi161.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGXadOoxwcYUpBOvDOHwRreOuQo+PCOSwcoQjSsc62RmCVySnCVdR6sXLE9o3/vznW6S63J92o/drQ/FMSVczVk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-16_15:45:14 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078 2023-12-16T21:27:28.293 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa; will attempt to use it 2023-12-16T21:27:28.294 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks 2023-12-16T21:27:28.294 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-16T21:27:28.295 INFO:teuthology.task.internal:Checking packages... 2023-12-16T21:27:28.320 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c7586e95519b7be76f3b6e3a4b333888baf46bc9' 2023-12-16T21:27:28.320 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-16T21:27:28.320 INFO:teuthology.packaging:ref: None 2023-12-16T21:27:28.320 INFO:teuthology.packaging:tag: None 2023-12-16T21:27:28.321 INFO:teuthology.packaging:branch: wip-yuri8-testing-2023-12-15-0911 2023-12-16T21:27:28.321 INFO:teuthology.packaging:sha1: c7586e95519b7be76f3b6e3a4b333888baf46bc9 2023-12-16T21:27:28.321 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri8-testing-2023-12-15-0911 2023-12-16T21:27:28.470 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-202-gc7586e95-1jammy 2023-12-16T21:27:28.471 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-16T21:27:28.479 INFO:teuthology.task.internal:no buildpackages task found 2023-12-16T21:27:28.479 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-16T21:27:28.489 INFO:teuthology.task.internal:Saving configuration 2023-12-16T21:27:28.505 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-16T21:27:28.512 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-16T21:27:28.540 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 21:21:49.522323', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM9CIvqYW0tllk5UWeuWujHXqBNdvaWsSuCh6ccaTsaQ'} 2023-12-16T21:27:28.562 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 21:21:49.524149', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGqTIzJ/yC9gn1iCNpCZKPpWVtubi3bLEe3WZeLsj2Vg'} 2023-12-16T21:27:28.563 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-16T21:27:28.571 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2023-12-16T21:27:28.571 INFO:teuthology.task.internal:roles: ubuntu@smithi161.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2023-12-16T21:27:28.571 INFO:teuthology.run_tasks:Running task console_log... 2023-12-16T21:27:28.651 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ffa88599b80>, signals=[15]) 2023-12-16T21:27:28.652 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-16T21:27:28.659 INFO:teuthology.task.internal:Opening connections... 2023-12-16T21:27:28.659 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:27:28.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:27:28.740 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:27:28.741 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:27:28.817 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-16T21:27:28.825 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2023-12-16T21:27:28.830 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2023-12-16T21:27:28.830 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2023-12-16T21:27:28.876 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-16T21:27:28.876 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="Ubuntu" 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="22.04" 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_CODENAME=jammy 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:ID=ubuntu 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE=debian 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-16T21:27:28.877 INFO:teuthology.orchestra.run.smithi046.stdout:UBUNTU_CODENAME=jammy 2023-12-16T21:27:28.878 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2023-12-16T21:27:28.897 DEBUG:teuthology.orchestra.run.smithi161:> uname -m 2023-12-16T21:27:28.902 INFO:teuthology.orchestra.run.smithi161.stdout:x86_64 2023-12-16T21:27:28.902 DEBUG:teuthology.orchestra.run.smithi161:> cat /etc/os-release 2023-12-16T21:27:28.949 INFO:teuthology.orchestra.run.smithi161.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-16T21:27:28.949 INFO:teuthology.orchestra.run.smithi161.stdout:NAME="Ubuntu" 2023-12-16T21:27:28.949 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_ID="22.04" 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:VERSION_CODENAME=jammy 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:ID=ubuntu 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:ID_LIKE=debian 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-16T21:27:28.950 INFO:teuthology.orchestra.run.smithi161.stdout:UBUNTU_CODENAME=jammy 2023-12-16T21:27:28.950 INFO:teuthology.lock.ops:Updating smithi161.front.sepia.ceph.com on lock server 2023-12-16T21:27:28.972 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-16T21:27:28.981 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-16T21:27:28.989 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-16T21:27:28.990 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2023-12-16T21:27:28.993 DEBUG:teuthology.orchestra.run.smithi161:> test '!' -e /home/ubuntu/cephtest 2023-12-16T21:27:28.998 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-16T21:27:29.006 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-16T21:27:29.007 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2023-12-16T21:27:29.041 DEBUG:teuthology.orchestra.run.smithi161:> test -z $(ls -A /var/lib/ceph) 2023-12-16T21:27:29.068 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-16T21:27:29.129 INFO:teuthology.run_tasks:Running task kernel... 2023-12-16T21:27:29.146 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-12-16T21:27:29.146 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-12-16T21:27:29.146 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-16T21:27:29.147 DEBUG:teuthology.orchestra.run.smithi161:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-16T21:27:29.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:27:29.151 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2023-12-16T21:27:29.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:27:29.153 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-12-16T21:27:29.200 INFO:teuthology.orchestra.run.smithi046.stdout:5.15.0-56-generic 2023-12-16T21:27:29.201 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.15.0-56-generic 2023-12-16T21:27:29.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get clean 2023-12-16T21:27:29.202 INFO:teuthology.orchestra.run.smithi161.stdout:5.15.0-56-generic 2023-12-16T21:27:29.202 INFO:teuthology.task.kernel:Running kernel on smithi161: 5.15.0-56-generic 2023-12-16T21:27:29.203 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get clean 2023-12-16T21:27:29.326 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2023-12-16T21:27:29.347 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2023-12-16T21:27:29.465 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-16T21:27:29.503 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-16T21:27:29.504 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-12-16T21:27:29.504 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-12-16T21:27:29.520 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-12-16T21:27:29.520 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-12-16T21:27:29.582 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-12-16T21:27:29.583 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-12-16T21:27:29.749 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,051 kB] 2023-12-16T21:27:29.757 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,051 kB] 2023-12-16T21:27:29.818 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,263 kB] 2023-12-16T21:27:29.848 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,263 kB] 2023-12-16T21:27:29.866 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [383 kB] 2023-12-16T21:27:29.868 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [383 kB] 2023-12-16T21:27:29.870 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [200 kB] 2023-12-16T21:27:29.873 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-12-16T21:27:29.873 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.4 kB] 2023-12-16T21:27:29.874 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,226 kB] 2023-12-16T21:27:29.879 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [200 kB] 2023-12-16T21:27:29.884 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-12-16T21:27:29.884 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.4 kB] 2023-12-16T21:27:29.885 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,226 kB] 2023-12-16T21:27:29.891 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [200 kB] 2023-12-16T21:27:29.897 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2023-12-16T21:27:29.897 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [823 kB] 2023-12-16T21:27:29.901 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [580 kB] 2023-12-16T21:27:29.905 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [156 kB] 2023-12-16T21:27:29.906 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-12-16T21:27:29.907 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-12-16T21:27:29.919 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [200 kB] 2023-12-16T21:27:29.925 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2023-12-16T21:27:29.925 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [823 kB] 2023-12-16T21:27:29.936 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [547 kB] 2023-12-16T21:27:29.936 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-12-16T21:27:29.936 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-12-16T21:27:29.936 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-12-16T21:27:29.941 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [580 kB] 2023-12-16T21:27:29.961 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [547 kB] 2023-12-16T21:27:29.975 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [260 kB] 2023-12-16T21:27:29.981 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [260 kB] 2023-12-16T21:27:29.981 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [156 kB] 2023-12-16T21:27:29.983 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2023-12-16T21:27:29.983 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2023-12-16T21:27:29.983 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-12-16T21:27:29.984 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-12-16T21:27:29.984 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.8 kB] 2023-12-16T21:27:29.985 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-12-16T21:27:29.985 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-12-16T21:27:29.986 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,250 kB] 2023-12-16T21:27:29.986 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2023-12-16T21:27:29.987 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.8 kB] 2023-12-16T21:27:29.988 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,250 kB] 2023-12-16T21:27:30.010 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [203 kB] 2023-12-16T21:27:30.013 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2023-12-16T21:27:30.014 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [675 kB] 2023-12-16T21:27:30.015 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [203 kB] 2023-12-16T21:27:30.020 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2023-12-16T21:27:30.020 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [675 kB] 2023-12-16T21:27:30.025 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,020 kB] 2023-12-16T21:27:30.034 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,020 kB] 2023-12-16T21:27:30.040 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [226 kB] 2023-12-16T21:27:30.044 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2023-12-16T21:27:30.044 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-12-16T21:27:30.045 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-12-16T21:27:30.046 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-12-16T21:27:30.057 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [226 kB] 2023-12-16T21:27:30.060 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2023-12-16T21:27:30.061 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-12-16T21:27:30.062 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-12-16T21:27:30.063 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-12-16T21:27:30.063 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-12-16T21:27:30.063 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2023-12-16T21:27:30.063 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2023-12-16T21:27:30.064 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-12-16T21:27:30.075 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-12-16T21:27:30.075 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2023-12-16T21:27:30.076 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2023-12-16T21:27:30.076 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-12-16T21:27:30.082 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-12-16T21:27:30.082 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2023-12-16T21:27:30.083 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2023-12-16T21:27:30.083 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2023-12-16T21:27:30.083 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-12-16T21:27:30.094 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-12-16T21:27:30.095 INFO:teuthology.orchestra.run.smithi161.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2023-12-16T21:27:30.095 INFO:teuthology.orchestra.run.smithi161.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2023-12-16T21:27:30.095 INFO:teuthology.orchestra.run.smithi161.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2023-12-16T21:27:30.096 INFO:teuthology.orchestra.run.smithi161.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2023-12-16T21:27:35.379 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 10.8 MB in 2s (6,576 kB/s) 2023-12-16T21:27:35.687 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 10.8 MB in 2s (6,388 kB/s) 2023-12-16T21:27:36.626 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2023-12-16T21:27:36.643 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-12-16T21:27:36.707 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2023-12-16T21:27:36.904 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2023-12-16T21:27:36.905 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2023-12-16T21:27:37.070 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:27:37.070 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:27:37.070 INFO:teuthology.orchestra.run.smithi161.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:27:37.070 INFO:teuthology.orchestra.run.smithi161.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:27:37.071 INFO:teuthology.orchestra.run.smithi161.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-12-16T21:27:37.071 INFO:teuthology.orchestra.run.smithi161.stdout: python2.7-minimal 2023-12-16T21:27:37.071 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:27:37.072 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2023-12-16T21:27:37.072 INFO:teuthology.orchestra.run.smithi161.stdout: linux-generic linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2023-12-16T21:27:37.072 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-generic linux-image-5.15.0-91-generic 2023-12-16T21:27:37.072 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-5.15.0-91-generic linux-modules-extra-5.15.0-91-generic 2023-12-16T21:27:37.073 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2023-12-16T21:27:37.073 INFO:teuthology.orchestra.run.smithi161.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-12-16T21:27:37.073 INFO:teuthology.orchestra.run.smithi161.stdout:Recommended packages: 2023-12-16T21:27:37.073 INFO:teuthology.orchestra.run.smithi161.stdout: thermald 2023-12-16T21:27:37.097 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-12-16T21:27:37.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-12-16T21:27:37.127 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2023-12-16T21:27:37.128 INFO:teuthology.orchestra.run.smithi161.stdout: linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2023-12-16T21:27:37.128 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-5.15.0-91-generic linux-modules-5.15.0-91-generic 2023-12-16T21:27:37.128 INFO:teuthology.orchestra.run.smithi161.stdout: linux-modules-extra-5.15.0-91-generic 2023-12-16T21:27:37.129 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2023-12-16T21:27:37.130 INFO:teuthology.orchestra.run.smithi161.stdout: linux-generic linux-headers-generic linux-image-generic 2023-12-16T21:27:37.177 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-12-16T21:27:37.310 INFO:teuthology.orchestra.run.smithi161.stdout:3 upgraded, 5 newly installed, 0 to remove and 277 not upgraded. 2023-12-16T21:27:37.310 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 113 MB of archives. 2023-12-16T21:27:37.310 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 582 MB of additional disk space will be used. 2023-12-16T21:27:37.311 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-91-generic amd64 5.15.0-91.101 [22.5 MB] 2023-12-16T21:27:37.372 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-12-16T21:27:37.372 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-12-16T21:27:37.545 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:27:37.546 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:27:37.546 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:27:37.546 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:27:37.546 INFO:teuthology.orchestra.run.smithi046.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-12-16T21:27:37.546 INFO:teuthology.orchestra.run.smithi046.stdout: python2.7-minimal 2023-12-16T21:27:37.546 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:27:37.547 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2023-12-16T21:27:37.547 INFO:teuthology.orchestra.run.smithi046.stdout: linux-generic linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2023-12-16T21:27:37.547 INFO:teuthology.orchestra.run.smithi046.stdout: linux-headers-generic linux-image-5.15.0-91-generic 2023-12-16T21:27:37.547 INFO:teuthology.orchestra.run.smithi046.stdout: linux-modules-5.15.0-91-generic linux-modules-extra-5.15.0-91-generic 2023-12-16T21:27:37.549 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2023-12-16T21:27:37.549 INFO:teuthology.orchestra.run.smithi046.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-12-16T21:27:37.549 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2023-12-16T21:27:37.549 INFO:teuthology.orchestra.run.smithi046.stdout: thermald 2023-12-16T21:27:37.602 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2023-12-16T21:27:37.602 INFO:teuthology.orchestra.run.smithi046.stdout: linux-headers-5.15.0-91 linux-headers-5.15.0-91-generic 2023-12-16T21:27:37.603 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.15.0-91-generic linux-modules-5.15.0-91-generic 2023-12-16T21:27:37.603 INFO:teuthology.orchestra.run.smithi046.stdout: linux-modules-extra-5.15.0-91-generic 2023-12-16T21:27:37.604 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2023-12-16T21:27:37.605 INFO:teuthology.orchestra.run.smithi046.stdout: linux-generic linux-headers-generic linux-image-generic 2023-12-16T21:27:37.659 INFO:teuthology.orchestra.run.smithi046.stdout:3 upgraded, 5 newly installed, 0 to remove and 277 not upgraded. 2023-12-16T21:27:37.660 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 113 MB of archives. 2023-12-16T21:27:37.660 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 582 MB of additional disk space will be used. 2023-12-16T21:27:37.660 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-91-generic amd64 5.15.0-91.101 [22.5 MB] 2023-12-16T21:27:38.090 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-91-generic amd64 5.15.0-91.101 [11.5 MB] 2023-12-16T21:27:38.201 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-91-generic amd64 5.15.0-91.101 [63.8 MB] 2023-12-16T21:27:38.696 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-91-generic amd64 5.15.0-91.101 [11.5 MB] 2023-12-16T21:27:39.030 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.91.88 [1,700 B] 2023-12-16T21:27:39.030 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.91.88 [2,488 B] 2023-12-16T21:27:39.030 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91 all 5.15.0-91.101 [12.3 MB] 2023-12-16T21:27:39.045 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-91-generic amd64 5.15.0-91.101 [63.8 MB] 2023-12-16T21:27:39.161 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91-generic amd64 5.15.0-91.101 [2,860 kB] 2023-12-16T21:27:39.260 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.91.88 [2,342 B] 2023-12-16T21:27:39.672 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 113 MB in 2s (68.5 MB/s) 2023-12-16T21:27:39.871 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-modules-5.15.0-91-generic. 2023-12-16T21:27:41.289 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.91.88 [1,700 B] 2023-12-16T21:27:41.289 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.91.88 [2,488 B] 2023-12-16T21:27:41.289 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91 all 5.15.0-91.101 [12.3 MB] 2023-12-16T21:27:41.982 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-12-16T21:27:41.986 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../0-linux-modules-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:27:42.092 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-91-generic amd64 5.15.0-91.101 [2,860 kB] 2023-12-16T21:27:42.144 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:27:42.215 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.91.88 [2,342 B] 2023-12-16T21:27:42.601 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 113 MB in 5s (22.3 MB/s) 2023-12-16T21:27:42.788 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-modules-5.15.0-91-generic. 2023-12-16T21:27:44.807 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-12-16T21:27:44.811 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../0-linux-modules-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:27:44.887 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-image-5.15.0-91-generic. 2023-12-16T21:27:44.891 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:27:44.909 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../1-linux-image-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:27:45.005 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:27:45.440 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-91-generic. 2023-12-16T21:27:45.461 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:27:45.489 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:27:47.333 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-image-5.15.0-91-generic. 2023-12-16T21:27:47.354 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../1-linux-image-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:27:47.440 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:27:47.936 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-91-generic. 2023-12-16T21:27:47.956 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:27:47.994 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:27:52.063 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../3-linux-generic_5.15.0.91.88_amd64.deb ... 2023-12-16T21:27:52.206 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2023-12-16T21:27:52.564 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.91.88_amd64.deb ... 2023-12-16T21:27:52.692 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-image-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2023-12-16T21:27:53.012 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-headers-5.15.0-91. 2023-12-16T21:27:53.035 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../5-linux-headers-5.15.0-91_5.15.0-91.101_all.deb ... 2023-12-16T21:27:53.070 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-5.15.0-91 (5.15.0-91.101) ... 2023-12-16T21:27:54.259 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../3-linux-generic_5.15.0.91.88_amd64.deb ... 2023-12-16T21:27:54.376 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2023-12-16T21:27:54.708 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.91.88_amd64.deb ... 2023-12-16T21:27:54.830 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-image-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2023-12-16T21:27:55.124 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-headers-5.15.0-91. 2023-12-16T21:27:55.146 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../5-linux-headers-5.15.0-91_5.15.0-91.101_all.deb ... 2023-12-16T21:27:55.182 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-5.15.0-91 (5.15.0-91.101) ... 2023-12-16T21:27:58.847 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package linux-headers-5.15.0-91-generic. 2023-12-16T21:27:58.884 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../6-linux-headers-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:27:58.922 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:00.917 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package linux-headers-5.15.0-91-generic. 2023-12-16T21:28:00.954 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../6-linux-headers-5.15.0-91-generic_5.15.0-91.101_amd64.deb ... 2023-12-16T21:28:00.992 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:02.418 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.91.88_amd64.deb ... 2023-12-16T21:28:02.557 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking linux-headers-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2023-12-16T21:28:02.907 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-5.15.0-91 (5.15.0-91.101) ... 2023-12-16T21:28:03.028 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:03.179 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-headers-generic (5.15.0.91.88) ... 2023-12-16T21:28:03.308 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:03.657 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.91.88_amd64.deb ... 2023-12-16T21:28:03.793 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking linux-headers-generic (5.15.0.91.88) over (5.15.0.56.54) ... 2023-12-16T21:28:04.135 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-5.15.0-91 (5.15.0-91.101) ... 2023-12-16T21:28:04.239 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:04.365 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-headers-generic (5.15.0.91.88) ... 2023-12-16T21:28:04.494 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-12-16T21:28:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-12-16T21:28:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-91-generic 2023-12-16T21:28:04.658 INFO:teuthology.orchestra.run.smithi046.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:05.193 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:05.813 INFO:teuthology.orchestra.run.smithi161.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-12-16T21:28:05.813 INFO:teuthology.orchestra.run.smithi161.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-12-16T21:28:05.813 INFO:teuthology.orchestra.run.smithi161.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-91-generic 2023-12-16T21:28:05.814 INFO:teuthology.orchestra.run.smithi161.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:05.954 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-modules-extra-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:06.886 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-image-generic (5.15.0.91.88) ... 2023-12-16T21:28:07.037 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:07.330 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-image-generic (5.15.0.91.88) ... 2023-12-16T21:28:07.473 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-modules-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:08.438 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up linux-generic (5.15.0.91.88) ... 2023-12-16T21:28:08.564 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:08.658 INFO:teuthology.orchestra.run.smithi046.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-12-16T21:28:08.658 INFO:teuthology.orchestra.run.smithi046.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:08.866 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up linux-generic (5.15.0.91.88) ... 2023-12-16T21:28:08.992 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for linux-image-5.15.0-91-generic (5.15.0-91.101) ... 2023-12-16T21:28:09.056 INFO:teuthology.orchestra.run.smithi161.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-12-16T21:28:09.056 INFO:teuthology.orchestra.run.smithi161.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:23.191 INFO:teuthology.orchestra.run.smithi046.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-12-16T21:28:23.191 INFO:teuthology.orchestra.run.smithi046.stdout:Sourcing file `/etc/default/grub' 2023-12-16T21:28:23.225 INFO:teuthology.orchestra.run.smithi046.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-16T21:28:23.243 INFO:teuthology.orchestra.run.smithi046.stdout:Generating grub configuration file ... 2023-12-16T21:28:23.538 INFO:teuthology.orchestra.run.smithi161.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-12-16T21:28:23.538 INFO:teuthology.orchestra.run.smithi161.stdout:Sourcing file `/etc/default/grub' 2023-12-16T21:28:23.564 INFO:teuthology.orchestra.run.smithi161.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-16T21:28:23.587 INFO:teuthology.orchestra.run.smithi161.stdout:Generating grub configuration file ... 2023-12-16T21:28:23.638 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2023-12-16T21:28:23.661 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:23.883 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2023-12-16T21:28:23.897 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:24.078 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-16T21:28:24.085 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-16T21:28:24.155 INFO:teuthology.orchestra.run.smithi046.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-16T21:28:24.161 INFO:teuthology.orchestra.run.smithi046.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-16T21:28:24.186 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-16T21:28:24.191 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-16T21:28:24.263 INFO:teuthology.orchestra.run.smithi161.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-16T21:28:24.268 INFO:teuthology.orchestra.run.smithi161.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-16T21:28:24.343 INFO:teuthology.orchestra.run.smithi046.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-16T21:28:24.344 INFO:teuthology.orchestra.run.smithi046.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-12-16T21:28:24.344 INFO:teuthology.orchestra.run.smithi046.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-16T21:28:24.381 INFO:teuthology.orchestra.run.smithi046.stdout:done 2023-12-16T21:28:24.457 INFO:teuthology.orchestra.run.smithi161.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-16T21:28:24.457 INFO:teuthology.orchestra.run.smithi161.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-12-16T21:28:24.457 INFO:teuthology.orchestra.run.smithi161.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-16T21:28:24.502 INFO:teuthology.orchestra.run.smithi161.stdout:done 2023-12-16T21:28:24.872 DEBUG:teuthology.orchestra.run.smithi046:> dpkg -s linux-image-generic 2023-12-16T21:28:24.897 INFO:teuthology.orchestra.run.smithi046.stdout:Package: linux-image-generic 2023-12-16T21:28:24.897 INFO:teuthology.orchestra.run.smithi046.stdout:Status: install ok installed 2023-12-16T21:28:24.897 INFO:teuthology.orchestra.run.smithi046.stdout:Priority: optional 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Section: kernel 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Installed-Size: 20 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Maintainer: Ubuntu Kernel Team 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: amd64 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Source: linux-meta 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Version: 5.15.0.91.88 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Recommends: thermald 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Description: Generic Linux kernel image 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout: This package will always depend on the latest generic kernel image 2023-12-16T21:28:24.898 INFO:teuthology.orchestra.run.smithi046.stdout: available. 2023-12-16T21:28:24.898 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-91-generic 2023-12-16T21:28:24.899 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-16T21:28:24.899 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-16T21:28:24.899 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-12-16T21:28:24.899 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2023-12-16T21:28:24.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get clean 2023-12-16T21:28:24.974 DEBUG:teuthology.orchestra.run.smithi161:> dpkg -s linux-image-generic 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Package: linux-image-generic 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Status: install ok installed 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Priority: optional 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Section: kernel 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Installed-Size: 20 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Maintainer: Ubuntu Kernel Team 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: amd64 2023-12-16T21:28:24.993 INFO:teuthology.orchestra.run.smithi161.stdout:Source: linux-meta 2023-12-16T21:28:24.994 INFO:teuthology.orchestra.run.smithi161.stdout:Version: 5.15.0.91.88 2023-12-16T21:28:24.994 INFO:teuthology.orchestra.run.smithi161.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-16T21:28:24.994 INFO:teuthology.orchestra.run.smithi161.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-16T21:28:24.994 INFO:teuthology.orchestra.run.smithi161.stdout:Recommends: thermald 2023-12-16T21:28:24.994 INFO:teuthology.orchestra.run.smithi161.stdout:Description: Generic Linux kernel image 2023-12-16T21:28:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: This package will always depend on the latest generic kernel image 2023-12-16T21:28:24.994 INFO:teuthology.orchestra.run.smithi161.stdout: available. 2023-12-16T21:28:24.994 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-91-generic 2023-12-16T21:28:24.994 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-12-16T21:28:24.994 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-12-16T21:28:24.994 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-12-16T21:28:24.994 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi161.front.sepia.ceph.com, path=None, version=distro) 2023-12-16T21:28:24.995 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get clean 2023-12-16T21:28:25.173 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2023-12-16T21:28:25.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2023-12-16T21:28:25.294 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-16T21:28:25.301 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-16T21:28:25.305 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-16T21:28:25.325 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-16T21:28:25.411 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-16T21:28:25.426 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-16T21:28:25.503 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-16T21:28:25.585 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-16T21:28:26.614 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2023-12-16T21:28:26.631 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-12-16T21:28:26.692 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2023-12-16T21:28:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-12-16T21:28:26.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-12-16T21:28:26.761 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-12-16T21:28:26.883 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2023-12-16T21:28:26.884 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2023-12-16T21:28:26.933 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-12-16T21:28:26.934 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-12-16T21:28:27.048 INFO:teuthology.orchestra.run.smithi161.stdout:linux-image-generic is already the newest version (5.15.0.91.88). 2023-12-16T21:28:27.049 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:28:27.049 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:28:27.049 INFO:teuthology.orchestra.run.smithi161.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:28:27.049 INFO:teuthology.orchestra.run.smithi161.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:28:27.049 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-16T21:28:27.049 INFO:teuthology.orchestra.run.smithi161.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-16T21:28:27.049 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:28:27.087 INFO:teuthology.orchestra.run.smithi046.stdout:linux-image-generic is already the newest version (5.15.0.91.88). 2023-12-16T21:28:27.087 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:28:27.087 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:28:27.087 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:28:27.087 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:28:27.087 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-16T21:28:27.087 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-16T21:28:27.088 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:28:27.090 INFO:teuthology.orchestra.run.smithi161.stdout:0 upgraded, 0 newly installed, 0 to remove and 277 not upgraded. 2023-12-16T21:28:27.093 DEBUG:teuthology.orchestra.run.smithi161:> dpkg -s linux-image-generic 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Package: linux-image-generic 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Status: install ok installed 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Priority: optional 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Section: kernel 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Installed-Size: 20 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Maintainer: Ubuntu Kernel Team 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Architecture: amd64 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Source: linux-meta 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Version: 5.15.0.91.88 2023-12-16T21:28:27.108 INFO:teuthology.orchestra.run.smithi161.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-16T21:28:27.109 INFO:teuthology.orchestra.run.smithi161.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-16T21:28:27.109 INFO:teuthology.orchestra.run.smithi161.stdout:Recommends: thermald 2023-12-16T21:28:27.109 INFO:teuthology.orchestra.run.smithi161.stdout:Description: Generic Linux kernel image 2023-12-16T21:28:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: This package will always depend on the latest generic kernel image 2023-12-16T21:28:27.109 INFO:teuthology.orchestra.run.smithi161.stdout: available. 2023-12-16T21:28:27.109 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-12-16T21:28:27.126 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 0 newly installed, 0 to remove and 277 not upgraded. 2023-12-16T21:28:27.129 DEBUG:teuthology.orchestra.run.smithi046:> dpkg -s linux-image-generic 2023-12-16T21:28:27.157 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.jcgErS3eyK 2023-12-16T21:28:27.157 DEBUG:teuthology.orchestra.run.smithi161:> sudo cp /boot/grub/grub.cfg /tmp/tmp.jcgErS3eyK 2023-12-16T21:28:27.183 INFO:teuthology.orchestra.run.smithi046.stdout:Package: linux-image-generic 2023-12-16T21:28:27.183 INFO:teuthology.orchestra.run.smithi046.stdout:Status: install ok installed 2023-12-16T21:28:27.183 INFO:teuthology.orchestra.run.smithi046.stdout:Priority: optional 2023-12-16T21:28:27.183 INFO:teuthology.orchestra.run.smithi046.stdout:Section: kernel 2023-12-16T21:28:27.183 INFO:teuthology.orchestra.run.smithi046.stdout:Installed-Size: 20 2023-12-16T21:28:27.183 INFO:teuthology.orchestra.run.smithi046.stdout:Maintainer: Ubuntu Kernel Team 2023-12-16T21:28:27.183 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: amd64 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout:Source: linux-meta 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout:Version: 5.15.0.91.88 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-91), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout:Depends: linux-image-5.15.0-91-generic, linux-modules-extra-5.15.0-91-generic, linux-firmware, intel-microcode, amd64-microcode 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout:Recommends: thermald 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout:Description: Generic Linux kernel image 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: This package will always depend on the latest generic kernel image 2023-12-16T21:28:27.184 INFO:teuthology.orchestra.run.smithi046.stdout: available. 2023-12-16T21:28:27.184 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-12-16T21:28:27.210 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.jcgErS3eyK 2023-12-16T21:28:27.228 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.93nJdXWLL7 2023-12-16T21:28:27.228 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub/grub.cfg /tmp/tmp.93nJdXWLL7 2023-12-16T21:28:27.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.93nJdXWLL7 2023-12-16T21:28:27.335 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.jcgErS3eyK is 10KB 2023-12-16T21:28:27.349 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.jcgErS3eyK 2023-12-16T21:28:27.354 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-12-16T21:28:27.409 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:28:27.410 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-12-16T21:28:27.410 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-12-16T21:28:27.412 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.93nJdXWLL7 is 10KB 2023-12-16T21:28:27.424 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.93nJdXWLL7 2023-12-16T21:28:27.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-12-16T21:28:27.483 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:28:27.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-12-16T21:28:27.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-12-16T21:28:27.495 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-91-generic 2023-12-16T21:28:27.495 DEBUG:teuthology.orchestra.run.smithi161:> sudo update-grub 2023-12-16T21:28:27.566 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-91-generic 2023-12-16T21:28:27.566 DEBUG:teuthology.orchestra.run.smithi046:> sudo update-grub 2023-12-16T21:28:28.652 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub' 2023-12-16T21:28:28.653 INFO:teuthology.orchestra.run.smithi161.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-16T21:28:28.655 INFO:teuthology.orchestra.run.smithi161.stderr:Generating grub configuration file ... 2023-12-16T21:28:28.836 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub' 2023-12-16T21:28:28.837 INFO:teuthology.orchestra.run.smithi046.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-16T21:28:28.838 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2023-12-16T21:28:28.840 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2023-12-16T21:28:28.844 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:29.060 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-91-generic 2023-12-16T21:28:29.066 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-91-generic 2023-12-16T21:28:29.082 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-16T21:28:29.088 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-16T21:28:29.130 INFO:teuthology.orchestra.run.smithi161.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-16T21:28:29.135 INFO:teuthology.orchestra.run.smithi161.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-16T21:28:29.306 INFO:teuthology.orchestra.run.smithi161.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-16T21:28:29.307 INFO:teuthology.orchestra.run.smithi161.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-16T21:28:29.307 INFO:teuthology.orchestra.run.smithi161.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-16T21:28:29.320 INFO:teuthology.orchestra.run.smithi161.stderr:done 2023-12-16T21:28:29.328 DEBUG:teuthology.orchestra.run.smithi161:> sudo shutdown -r now 2023-12-16T21:28:29.458 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-16T21:28:29.464 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-16T21:28:29.508 INFO:teuthology.orchestra.run.smithi046.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-16T21:28:29.513 INFO:teuthology.orchestra.run.smithi046.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-16T21:28:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-16T21:28:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-16T21:28:29.636 INFO:teuthology.orchestra.run.smithi046.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-16T21:28:29.652 INFO:teuthology.orchestra.run.smithi046.stderr:done 2023-12-16T21:28:29.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2023-12-16T21:28:59.359 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-16T21:28:59.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:28:59.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:28:59.656 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-16T21:28:59.657 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-16T21:28:59.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:29:17.842 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-12-16T21:29:18.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-12-16T21:29:26.851 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:29:26.852 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:29:27.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-16T21:29:27.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:29:29.905 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-12-16T21:29:30.161 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2023-12-16T21:29:41.911 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:29:41.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:29:42.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-16T21:29:42.163 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:30:00.338 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-12-16T21:30:15.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:30:15.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:30:15.356 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.161 2023-12-16T21:30:33.373 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:30:33.374 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:30:33.613 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-12-16T21:30:34.204 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:30:34.204 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-91-generic"... 2023-12-16T21:30:34.204 DEBUG:teuthology.orchestra.run.smithi161:> uname -r 2023-12-16T21:30:34.252 INFO:teuthology.orchestra.run.smithi161.stdout:5.15.0-91-generic 2023-12-16T21:30:34.252 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-91-generic vs 5.15.0-91-generic 2023-12-16T21:30:34.252 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-16T21:30:34.252 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-16T21:30:35.253 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-16T21:30:35.254 DEBUG:teuthology.orchestra.run.smithi161:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-16T21:30:35.614 INFO:teuthology.orchestra.run.smithi161.stdout:ttyS1 2023-12-16T21:30:35.646 DEBUG:teuthology.parallel:result is None 2023-12-16T21:30:42.165 DEBUG:teuthology.orchestra.remote:timed out 2023-12-16T21:30:57.166 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-16T21:30:57.167 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:30:57.437 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-12-16T21:30:58.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-16T21:30:58.077 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-91-generic"... 2023-12-16T21:30:58.077 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2023-12-16T21:30:58.125 INFO:teuthology.orchestra.run.smithi046.stdout:5.15.0-91-generic 2023-12-16T21:30:58.125 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-91-generic vs 5.15.0-91-generic 2023-12-16T21:30:58.125 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-16T21:30:58.125 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-16T21:30:59.126 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-16T21:30:59.126 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-16T21:30:59.191 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2023-12-16T21:30:59.218 DEBUG:teuthology.parallel:result is None 2023-12-16T21:30:59.219 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-16T21:30:59.225 INFO:teuthology.task.internal:Creating test directory... 2023-12-16T21:30:59.226 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-16T21:30:59.228 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-16T21:30:59.234 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-16T21:30:59.283 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-16T21:30:59.309 INFO:teuthology.task.internal:Creating archive directory... 2023-12-16T21:30:59.309 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-16T21:30:59.312 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-16T21:30:59.403 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-16T21:30:59.410 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-16T21:30:59.410 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-16T21:30:59.413 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-16T21:30:59.430 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-16T21:30:59.432 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-16T21:30:59.438 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-16T21:30:59.441 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-16T21:30:59.442 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-16T21:30:59.449 INFO:teuthology.task.internal:Configuring sudo... 2023-12-16T21:30:59.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-16T21:30:59.490 DEBUG:teuthology.orchestra.run.smithi161:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-16T21:30:59.502 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-16T21:30:59.536 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-16T21:30:59.536 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-16T21:30:59.546 DEBUG:teuthology.orchestra.run.smithi161:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-16T21:30:59.551 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-16T21:30:59.597 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-16T21:30:59.645 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:30:59.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-16T21:30:59.703 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-16T21:30:59.709 DEBUG:teuthology.orchestra.run.smithi161:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-16T21:30:59.757 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:30:59.757 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-16T21:30:59.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2023-12-16T21:30:59.815 DEBUG:teuthology.orchestra.run.smithi161:> sudo service rsyslog restart 2023-12-16T21:30:59.896 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-16T21:30:59.906 INFO:teuthology.task.internal:Starting timer... 2023-12-16T21:30:59.906 INFO:teuthology.run_tasks:Running task pcp... 2023-12-16T21:30:59.923 INFO:teuthology.run_tasks:Running task selinux... 2023-12-16T21:30:59.938 DEBUG:teuthology.task.selinux:Excluding smithi046: OS 'ubuntu' does not support SELinux 2023-12-16T21:30:59.938 DEBUG:teuthology.task.selinux:Excluding smithi161: OS 'ubuntu' does not support SELinux 2023-12-16T21:30:59.938 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-16T21:30:59.938 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-12-16T21:30:59.938 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-16T21:30:59.938 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-16T21:30:59.957 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-16T21:30:59.963 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-16T21:31:00.115 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-16T21:31:00.138 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-16T21:31:00.139 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi161.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-16T21:38:53.886 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi161.front.sepia.ceph.com')] 2023-12-16T21:38:53.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-16T21:38:53.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:38:53.965 DEBUG:teuthology.orchestra.run.smithi046:> true 2023-12-16T21:38:54.045 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2023-12-16T21:38:54.046 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:38:54.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi161.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-16T21:38:54.124 DEBUG:teuthology.orchestra.run.smithi161:> true 2023-12-16T21:38:54.197 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi161.front.sepia.ceph.com' 2023-12-16T21:38:54.198 INFO:teuthology.run_tasks:Running task clock... 2023-12-16T21:38:54.211 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-16T21:38:54.211 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-16T21:38:54.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-16T21:38:54.215 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-16T21:38:54.215 DEBUG:teuthology.orchestra.run.smithi161:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Command line: ntpd -gq 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: ---------------------------------------------------- 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: ntp-4 is maintained by Network Time Foundation, 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: corporation. Support and training for ntp-4 are 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: available at https://www.nwtime.org/support 2023-12-16T21:38:54.243 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: ---------------------------------------------------- 2023-12-16T21:38:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: proto: precision = 0.066 usec (-24) 2023-12-16T21:38:54.245 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: basedate set to 2022-02-04 2023-12-16T21:38:54.245 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: gps base set to 2022-02-06 (week 2196) 2023-12-16T21:38:54.246 INFO:teuthology.orchestra.run.smithi046.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-16T21:38:54.248 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-16T21:38:54.248 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: restrict ::: KOD does nothing without LIMITED. 2023-12-16T21:38:54.248 INFO:teuthology.orchestra.run.smithi046.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-16T21:38:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Listen and drop on 0 v6wildcard [::]:123 2023-12-16T21:38:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-16T21:38:54.250 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Listen normally on 2 lo 127.0.0.1:123 2023-12-16T21:38:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Listen normally on 3 ens1f0 172.21.15.46:123 2023-12-16T21:38:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Listen normally on 4 lo [::1]:123 2023-12-16T21:38:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:15f8%4]:123 2023-12-16T21:38:54.251 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:38:54 ntpd[27955]: Listening on routing socket on fd #22 for interface updates 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Command line: ntpd -gq 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: ---------------------------------------------------- 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: ntp-4 is maintained by Network Time Foundation, 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: corporation. Support and training for ntp-4 are 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: available at https://www.nwtime.org/support 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: ---------------------------------------------------- 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: proto: precision = 0.044 usec (-24) 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: basedate set to 2022-02-04 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: gps base set to 2022-02-06 (week 2196) 2023-12-16T21:38:54.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-16T21:38:54.271 INFO:teuthology.orchestra.run.smithi161.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-16T21:38:54.271 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: restrict ::: KOD does nothing without LIMITED. 2023-12-16T21:38:54.271 INFO:teuthology.orchestra.run.smithi161.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-16T21:38:54.272 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Listen and drop on 0 v6wildcard [::]:123 2023-12-16T21:38:54.272 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-16T21:38:54.272 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Listen normally on 2 lo 127.0.0.1:123 2023-12-16T21:38:54.272 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Listen normally on 3 enp3s0f1 172.21.15.161:123 2023-12-16T21:38:54.272 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Listen normally on 4 lo [::1]:123 2023-12-16T21:38:54.272 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7251%5]:123 2023-12-16T21:38:54.272 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:38:54 ntpd[28036]: Listening on routing socket on fd #22 for interface updates 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:55.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:55.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:55.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:55 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:55.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:55 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:56 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:56 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:56 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:56 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:56 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:56.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:56 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:56.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:56.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:56.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:56.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:56.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:56.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:56.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:56.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:56.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:56.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:56.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:56.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:56 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:57.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:57 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:57 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:57 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:57 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:57 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:57 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:57.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:57 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:58 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:58 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:58 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:58.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:58 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:58 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:58.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:58 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:58.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:58.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:58 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:59.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:59.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:59.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:38:59 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:38:59.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:59 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:59.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:59 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:59.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:59 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:38:59.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:59 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:38:59.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:59 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:38:59.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:38:59 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:39:00.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:00 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:00.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:00 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:00.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:00 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:00.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:00 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:00.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:00 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:39:00.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:00 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:39:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:00.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:00.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:39:00.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:39:00.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:00.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:00.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:39:00.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:00 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:39:01.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:01.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:01.243 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:39:01.244 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:39:01.244 INFO:teuthology.orchestra.run.smithi046.stdout:16 Dec 21:39:01 ntpd[27955]: ntpd: time slew -0.000377 s 2023-12-16T21:39:01.244 INFO:teuthology.orchestra.run.smithi046.stdout:ntpd: time slew -0.000377s 2023-12-16T21:39:01.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-16T21:39:01.245 INFO:teuthology.orchestra.run.smithi046.stderr:16 Dec 21:39:01 ntpd[27955]: can't open /var/log/ntpstats/loopstats.20231216: Permission denied 2023-12-16T21:39:01.269 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:01.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:01.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-16T21:39:01.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: can't open /var/log/ntpstats/rawstats.20231216: Permission denied 2023-12-16T21:39:01.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-16T21:39:01.270 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: can't open /var/log/ntpstats/peerstats.20231216: Permission denied 2023-12-16T21:39:01.270 INFO:teuthology.orchestra.run.smithi161.stdout:16 Dec 21:39:01 ntpd[28036]: ntpd: time slew -0.001004 s 2023-12-16T21:39:01.270 INFO:teuthology.orchestra.run.smithi161.stdout:ntpd: time slew -0.001004s 2023-12-16T21:39:01.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-16T21:39:01.271 INFO:teuthology.orchestra.run.smithi161.stderr:16 Dec 21:39:01 ntpd[28036]: can't open /var/log/ntpstats/loopstats.20231216: Permission denied 2023-12-16T21:39:01.309 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2023-12-16T21:39:01.309 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2023-12-16T21:39:01.309 INFO:teuthology.orchestra.run.smithi046.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.309 INFO:teuthology.orchestra.run.smithi046.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.309 INFO:teuthology.orchestra.run.smithi046.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.309 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.318 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2023-12-16T21:39:01.318 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2023-12-16T21:39:01.318 INFO:teuthology.orchestra.run.smithi161.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.318 INFO:teuthology.orchestra.run.smithi161.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.319 INFO:teuthology.orchestra.run.smithi161.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.319 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:39:01.320 INFO:teuthology.run_tasks:Running task install... 2023-12-16T21:39:01.329 DEBUG:teuthology.task.install:project ceph 2023-12-16T21:39:01.329 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9'}} 2023-12-16T21:39:01.330 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9'} 2023-12-16T21:39:01.330 INFO:teuthology.task.install:Using flavor: default 2023-12-16T21:39:01.337 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-16T21:39:01.337 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2023-12-16T21:39:01.338 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-key list | grep Ceph 2023-12-16T21:39:01.338 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-key list | grep Ceph 2023-12-16T21:39:01.412 INFO:teuthology.orchestra.run.smithi046.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-12-16T21:39:01.416 INFO:teuthology.orchestra.run.smithi161.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-12-16T21:39:01.443 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-12-16T21:39:01.443 INFO:teuthology.orchestra.run.smithi046.stdout:uid [ unknown] Ceph.com (release key) 2023-12-16T21:39:01.444 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-12-16T21:39:01.444 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-12-16T21:39:01.444 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7586e95519b7be76f3b6e3a4b333888baf46bc9 2023-12-16T21:39:01.449 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-12-16T21:39:01.449 INFO:teuthology.orchestra.run.smithi161.stdout:uid [ unknown] Ceph.com (release key) 2023-12-16T21:39:01.449 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-12-16T21:39:01.449 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-12-16T21:39:01.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7586e95519b7be76f3b6e3a4b333888baf46bc9 2023-12-16T21:39:01.586 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default/ 2023-12-16T21:39:01.587 INFO:teuthology.task.install.deb:Package version is 19.0.0-202-gc7586e95-1jammy 2023-12-16T21:39:01.669 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default/ 2023-12-16T21:39:01.669 INFO:teuthology.task.install.deb:Package version is 19.0.0-202-gc7586e95-1jammy 2023-12-16T21:39:01.766 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:39:01.766 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-12-16T21:39:01.774 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:39:01.774 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-12-16T21:39:01.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo apt-get update 2023-12-16T21:39:01.789 DEBUG:teuthology.orchestra.run.smithi161:> sudo apt-get update 2023-12-16T21:39:01.950 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-16T21:39:01.950 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-16T21:39:01.952 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-16T21:39:01.952 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-16T21:39:01.958 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-16T21:39:01.962 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-16T21:39:01.975 INFO:teuthology.orchestra.run.smithi046.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-16T21:39:01.979 INFO:teuthology.orchestra.run.smithi161.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-16T21:39:02.006 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy InRelease 2023-12-16T21:39:02.007 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy InRelease 2023-12-16T21:39:02.027 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-12-16T21:39:02.031 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-12-16T21:39:02.049 INFO:teuthology.orchestra.run.smithi046.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy Release.gpg 2023-12-16T21:39:02.052 INFO:teuthology.orchestra.run.smithi161.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy Release.gpg 2023-12-16T21:39:02.070 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,390 B] 2023-12-16T21:39:02.073 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,390 B] 2023-12-16T21:39:02.096 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-12-16T21:39:02.099 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-12-16T21:39:02.297 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 28.5 kB in 0s (86.6 kB/s) 2023-12-16T21:39:02.310 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 28.5 kB in 0s (82.2 kB/s) 2023-12-16T21:39:03.226 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2023-12-16T21:39:03.229 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-12-16T21:39:03.242 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-202-gc7586e95-1jammy cephadm=19.0.0-202-gc7586e95-1jammy ceph-mds=19.0.0-202-gc7586e95-1jammy ceph-mgr=19.0.0-202-gc7586e95-1jammy ceph-common=19.0.0-202-gc7586e95-1jammy ceph-fuse=19.0.0-202-gc7586e95-1jammy ceph-test=19.0.0-202-gc7586e95-1jammy ceph-volume=19.0.0-202-gc7586e95-1jammy radosgw=19.0.0-202-gc7586e95-1jammy python3-rados=19.0.0-202-gc7586e95-1jammy python3-rgw=19.0.0-202-gc7586e95-1jammy python3-cephfs=19.0.0-202-gc7586e95-1jammy python3-rbd=19.0.0-202-gc7586e95-1jammy libcephfs2=19.0.0-202-gc7586e95-1jammy libcephfs-dev=19.0.0-202-gc7586e95-1jammy librados2=19.0.0-202-gc7586e95-1jammy librbd1=19.0.0-202-gc7586e95-1jammy rbd-fuse=19.0.0-202-gc7586e95-1jammy python3-cephfs=19.0.0-202-gc7586e95-1jammy cephfs-shell=19.0.0-202-gc7586e95-1jammy cephfs-top=19.0.0-202-gc7586e95-1jammy cephfs-mirror=19.0.0-202-gc7586e95-1jammy 2023-12-16T21:39:03.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-202-gc7586e95-1jammy cephadm=19.0.0-202-gc7586e95-1jammy ceph-mds=19.0.0-202-gc7586e95-1jammy ceph-mgr=19.0.0-202-gc7586e95-1jammy ceph-common=19.0.0-202-gc7586e95-1jammy ceph-fuse=19.0.0-202-gc7586e95-1jammy ceph-test=19.0.0-202-gc7586e95-1jammy ceph-volume=19.0.0-202-gc7586e95-1jammy radosgw=19.0.0-202-gc7586e95-1jammy python3-rados=19.0.0-202-gc7586e95-1jammy python3-rgw=19.0.0-202-gc7586e95-1jammy python3-cephfs=19.0.0-202-gc7586e95-1jammy python3-rbd=19.0.0-202-gc7586e95-1jammy libcephfs2=19.0.0-202-gc7586e95-1jammy libcephfs-dev=19.0.0-202-gc7586e95-1jammy librados2=19.0.0-202-gc7586e95-1jammy librbd1=19.0.0-202-gc7586e95-1jammy rbd-fuse=19.0.0-202-gc7586e95-1jammy python3-cephfs=19.0.0-202-gc7586e95-1jammy cephfs-shell=19.0.0-202-gc7586e95-1jammy cephfs-top=19.0.0-202-gc7586e95-1jammy cephfs-mirror=19.0.0-202-gc7586e95-1jammy 2023-12-16T21:39:03.297 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-12-16T21:39:03.303 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2023-12-16T21:39:03.468 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-12-16T21:39:03.468 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-12-16T21:39:03.492 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2023-12-16T21:39:03.493 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2023-12-16T21:39:03.626 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:39:03.626 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:39:03.626 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:39:03.626 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:39:03.627 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-16T21:39:03.627 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-16T21:39:03.627 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:39:03.628 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2023-12-16T21:39:03.628 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-12-16T21:39:03.628 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-12-16T21:39:03.628 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-12-16T21:39:03.628 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-12-16T21:39:03.629 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-12-16T21:39:03.630 INFO:teuthology.orchestra.run.smithi046.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout:Recommended packages: 2023-12-16T21:39:03.631 INFO:teuthology.orchestra.run.smithi046.stdout: btrfs-tools 2023-12-16T21:39:03.661 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:39:03.661 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:39:03.661 INFO:teuthology.orchestra.run.smithi161.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:39:03.661 INFO:teuthology.orchestra.run.smithi161.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:39:03.661 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-16T21:39:03.662 INFO:teuthology.orchestra.run.smithi161.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-16T21:39:03.662 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:39:03.662 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2023-12-16T21:39:03.662 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-12-16T21:39:03.662 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-12-16T21:39:03.663 INFO:teuthology.orchestra.run.smithi161.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-12-16T21:39:03.663 INFO:teuthology.orchestra.run.smithi161.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-12-16T21:39:03.663 INFO:teuthology.orchestra.run.smithi161.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-12-16T21:39:03.664 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2023-12-16T21:39:03.665 INFO:teuthology.orchestra.run.smithi161.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2023-12-16T21:39:03.666 INFO:teuthology.orchestra.run.smithi161.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2023-12-16T21:39:03.666 INFO:teuthology.orchestra.run.smithi161.stdout:Recommended packages: 2023-12-16T21:39:03.666 INFO:teuthology.orchestra.run.smithi161.stdout: btrfs-tools 2023-12-16T21:39:03.737 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2023-12-16T21:39:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-12-16T21:39:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-12-16T21:39:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-12-16T21:39:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-12-16T21:39:03.737 INFO:teuthology.orchestra.run.smithi046.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-12-16T21:39:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-12-16T21:39:03.738 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-12-16T21:39:03.739 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-12-16T21:39:03.740 INFO:teuthology.orchestra.run.smithi046.stdout: socat unzip xmlstarlet zip 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-12-16T21:39:03.770 INFO:teuthology.orchestra.run.smithi161.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2023-12-16T21:39:03.771 INFO:teuthology.orchestra.run.smithi161.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-12-16T21:39:03.772 INFO:teuthology.orchestra.run.smithi161.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-12-16T21:39:03.773 INFO:teuthology.orchestra.run.smithi161.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-12-16T21:39:03.773 INFO:teuthology.orchestra.run.smithi161.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-12-16T21:39:03.773 INFO:teuthology.orchestra.run.smithi161.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-12-16T21:39:03.773 INFO:teuthology.orchestra.run.smithi161.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-12-16T21:39:03.773 INFO:teuthology.orchestra.run.smithi161.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-12-16T21:39:03.773 INFO:teuthology.orchestra.run.smithi161.stdout: socat unzip xmlstarlet zip 2023-12-16T21:39:03.802 INFO:teuthology.orchestra.run.smithi046.stdout:0 upgraded, 124 newly installed, 0 to remove and 265 not upgraded. 2023-12-16T21:39:03.802 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 187 MB of archives. 2023-12-16T21:39:03.802 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 835 MB of additional disk space will be used. 2023-12-16T21:39:03.802 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-12-16T21:39:03.823 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-12-16T21:39:03.829 INFO:teuthology.orchestra.run.smithi161.stdout:0 upgraded, 124 newly installed, 0 to remove and 265 not upgraded. 2023-12-16T21:39:03.829 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 187 MB of archives. 2023-12-16T21:39:03.829 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 835 MB of additional disk space will be used. 2023-12-16T21:39:03.829 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-12-16T21:39:03.833 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-12-16T21:39:03.847 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-12-16T21:39:03.857 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-12-16T21:39:03.881 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-12-16T21:39:03.896 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-12-16T21:39:03.905 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-12-16T21:39:03.909 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-12-16T21:39:03.914 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-202-gc7586e95-1jammy [3,460 kB] 2023-12-16T21:39:03.917 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-12-16T21:39:03.920 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-12-16T21:39:03.921 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-12-16T21:39:03.931 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-12-16T21:39:03.958 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-202-gc7586e95-1jammy [3,460 kB] 2023-12-16T21:39:03.973 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-12-16T21:39:03.977 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-12-16T21:39:03.978 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-12-16T21:39:03.982 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-12-16T21:39:03.989 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-12-16T21:39:03.993 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-12-16T21:39:03.994 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-12-16T21:39:03.994 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-12-16T21:39:03.997 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-12-16T21:39:03.998 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-12-16T21:39:04.002 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2023-12-16T21:39:04.004 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2023-12-16T21:39:04.006 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2023-12-16T21:39:04.007 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2023-12-16T21:39:04.008 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2023-12-16T21:39:04.008 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2023-12-16T21:39:04.008 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2023-12-16T21:39:04.010 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2023-12-16T21:39:04.011 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2023-12-16T21:39:04.011 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2023-12-16T21:39:04.015 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2023-12-16T21:39:04.017 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.1 [174 kB] 2023-12-16T21:39:04.017 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2023-12-16T21:39:04.019 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2023-12-16T21:39:04.021 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-12-16T21:39:04.031 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-12-16T21:39:04.032 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-12-16T21:39:04.032 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-12-16T21:39:04.033 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-12-16T21:39:04.033 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-12-16T21:39:04.033 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-12-16T21:39:04.034 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-12-16T21:39:04.034 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-12-16T21:39:04.036 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-12-16T21:39:04.037 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-12-16T21:39:04.038 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-12-16T21:39:04.038 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-12-16T21:39:04.038 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-12-16T21:39:04.039 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-12-16T21:39:04.039 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-12-16T21:39:04.049 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-12-16T21:39:04.049 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-12-16T21:39:04.057 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-12-16T21:39:04.057 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-12-16T21:39:04.059 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-12-16T21:39:04.065 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-12-16T21:39:04.070 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-12-16T21:39:04.070 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-12-16T21:39:04.070 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-12-16T21:39:04.070 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-12-16T21:39:04.070 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-12-16T21:39:04.070 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-12-16T21:39:04.070 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-12-16T21:39:04.076 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-12-16T21:39:04.076 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-12-16T21:39:04.077 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-12-16T21:39:04.077 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-12-16T21:39:04.078 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-12-16T21:39:04.078 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-12-16T21:39:04.079 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-12-16T21:39:04.082 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-12-16T21:39:04.083 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-12-16T21:39:04.096 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-12-16T21:39:04.096 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-12-16T21:39:04.099 INFO:teuthology.orchestra.run.smithi161.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-12-16T21:39:04.099 INFO:teuthology.orchestra.run.smithi161.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-12-16T21:39:04.100 INFO:teuthology.orchestra.run.smithi161.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-12-16T21:39:04.100 INFO:teuthology.orchestra.run.smithi161.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-12-16T21:39:04.101 INFO:teuthology.orchestra.run.smithi046.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-12-16T21:39:04.101 INFO:teuthology.orchestra.run.smithi161.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-12-16T21:39:04.103 INFO:teuthology.orchestra.run.smithi161.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-12-16T21:39:04.104 INFO:teuthology.orchestra.run.smithi161.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-12-16T21:39:04.105 INFO:teuthology.orchestra.run.smithi161.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-12-16T21:39:04.115 INFO:teuthology.orchestra.run.smithi046.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-12-16T21:39:04.117 INFO:teuthology.orchestra.run.smithi161.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-12-16T21:39:04.117 INFO:teuthology.orchestra.run.smithi046.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-12-16T21:39:04.117 INFO:teuthology.orchestra.run.smithi046.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-12-16T21:39:04.117 INFO:teuthology.orchestra.run.smithi046.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-12-16T21:39:04.118 INFO:teuthology.orchestra.run.smithi161.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-12-16T21:39:04.119 INFO:teuthology.orchestra.run.smithi046.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-12-16T21:39:04.120 INFO:teuthology.orchestra.run.smithi046.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-12-16T21:39:04.121 INFO:teuthology.orchestra.run.smithi046.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-12-16T21:39:04.133 INFO:teuthology.orchestra.run.smithi161.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-12-16T21:39:04.133 INFO:teuthology.orchestra.run.smithi161.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-12-16T21:39:04.135 INFO:teuthology.orchestra.run.smithi046.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-12-16T21:39:04.135 INFO:teuthology.orchestra.run.smithi161.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-12-16T21:39:04.137 INFO:teuthology.orchestra.run.smithi046.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-12-16T21:39:04.138 INFO:teuthology.orchestra.run.smithi161.stdout:Get:54 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-202-gc7586e95-1jammy [3,109 kB] 2023-12-16T21:39:04.138 INFO:teuthology.orchestra.run.smithi046.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-12-16T21:39:04.155 INFO:teuthology.orchestra.run.smithi046.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-12-16T21:39:04.157 INFO:teuthology.orchestra.run.smithi046.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-12-16T21:39:04.186 INFO:teuthology.orchestra.run.smithi161.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-12-16T21:39:04.187 INFO:teuthology.orchestra.run.smithi161.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-12-16T21:39:04.188 INFO:teuthology.orchestra.run.smithi161.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-12-16T21:39:04.188 INFO:teuthology.orchestra.run.smithi161.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-12-16T21:39:04.189 INFO:teuthology.orchestra.run.smithi161.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-12-16T21:39:04.206 INFO:teuthology.orchestra.run.smithi161.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-12-16T21:39:04.207 INFO:teuthology.orchestra.run.smithi161.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-12-16T21:39:04.208 INFO:teuthology.orchestra.run.smithi161.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-12-16T21:39:04.210 INFO:teuthology.orchestra.run.smithi046.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-12-16T21:39:04.211 INFO:teuthology.orchestra.run.smithi046.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-12-16T21:39:04.212 INFO:teuthology.orchestra.run.smithi046.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-12-16T21:39:04.212 INFO:teuthology.orchestra.run.smithi046.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-12-16T21:39:04.213 INFO:teuthology.orchestra.run.smithi046.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-12-16T21:39:04.222 INFO:teuthology.orchestra.run.smithi161.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-12-16T21:39:04.222 INFO:teuthology.orchestra.run.smithi161.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-12-16T21:39:04.223 INFO:teuthology.orchestra.run.smithi161.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-12-16T21:39:04.223 INFO:teuthology.orchestra.run.smithi161.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-12-16T21:39:04.223 INFO:teuthology.orchestra.run.smithi161.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-12-16T21:39:04.224 INFO:teuthology.orchestra.run.smithi161.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-12-16T21:39:04.231 INFO:teuthology.orchestra.run.smithi046.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-12-16T21:39:04.233 INFO:teuthology.orchestra.run.smithi046.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-12-16T21:39:04.234 INFO:teuthology.orchestra.run.smithi046.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-12-16T21:39:04.237 INFO:teuthology.orchestra.run.smithi161.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-12-16T21:39:04.238 INFO:teuthology.orchestra.run.smithi161.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-12-16T21:39:04.241 INFO:teuthology.orchestra.run.smithi161.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-12-16T21:39:04.248 INFO:teuthology.orchestra.run.smithi046.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-12-16T21:39:04.249 INFO:teuthology.orchestra.run.smithi046.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-12-16T21:39:04.254 INFO:teuthology.orchestra.run.smithi161.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-12-16T21:39:04.255 INFO:teuthology.orchestra.run.smithi161.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-12-16T21:39:04.256 INFO:teuthology.orchestra.run.smithi161.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-12-16T21:39:04.257 INFO:teuthology.orchestra.run.smithi046.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-12-16T21:39:04.258 INFO:teuthology.orchestra.run.smithi161.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-202-gc7586e95-1jammy [690 kB] 2023-12-16T21:39:04.258 INFO:teuthology.orchestra.run.smithi046.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-12-16T21:39:04.259 INFO:teuthology.orchestra.run.smithi046.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-12-16T21:39:04.259 INFO:teuthology.orchestra.run.smithi046.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-12-16T21:39:04.259 INFO:teuthology.orchestra.run.smithi046.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-12-16T21:39:04.260 INFO:teuthology.orchestra.run.smithi161.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-12-16T21:39:04.260 INFO:teuthology.orchestra.run.smithi046.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-12-16T21:39:04.261 INFO:teuthology.orchestra.run.smithi161.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-12-16T21:39:04.261 INFO:teuthology.orchestra.run.smithi161.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2023-12-16T21:39:04.262 INFO:teuthology.orchestra.run.smithi161.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2023-12-16T21:39:04.263 INFO:teuthology.orchestra.run.smithi046.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-12-16T21:39:04.273 INFO:teuthology.orchestra.run.smithi161.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-12-16T21:39:04.275 INFO:teuthology.orchestra.run.smithi161.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2023-12-16T21:39:04.275 INFO:teuthology.orchestra.run.smithi161.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-12-16T21:39:04.276 INFO:teuthology.orchestra.run.smithi046.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-12-16T21:39:04.278 INFO:teuthology.orchestra.run.smithi046.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-12-16T21:39:04.279 INFO:teuthology.orchestra.run.smithi046.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-12-16T21:39:04.286 INFO:teuthology.orchestra.run.smithi161.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-202-gc7586e95-1jammy [356 kB] 2023-12-16T21:39:04.287 INFO:teuthology.orchestra.run.smithi161.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-12-16T21:39:04.288 INFO:teuthology.orchestra.run.smithi161.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-12-16T21:39:04.288 INFO:teuthology.orchestra.run.smithi161.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-12-16T21:39:04.289 INFO:teuthology.orchestra.run.smithi161.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-12-16T21:39:04.290 INFO:teuthology.orchestra.run.smithi161.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-12-16T21:39:04.291 INFO:teuthology.orchestra.run.smithi161.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-12-16T21:39:04.296 INFO:teuthology.orchestra.run.smithi046.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-12-16T21:39:04.297 INFO:teuthology.orchestra.run.smithi161.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-12-16T21:39:04.299 INFO:teuthology.orchestra.run.smithi046.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-12-16T21:39:04.299 INFO:teuthology.orchestra.run.smithi046.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2023-12-16T21:39:04.300 INFO:teuthology.orchestra.run.smithi046.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2023-12-16T21:39:04.301 INFO:teuthology.orchestra.run.smithi161.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-202-gc7586e95-1jammy [32.8 kB] 2023-12-16T21:39:04.301 INFO:teuthology.orchestra.run.smithi046.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-12-16T21:39:04.302 INFO:teuthology.orchestra.run.smithi161.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-202-gc7586e95-1jammy [180 kB] 2023-12-16T21:39:04.304 INFO:teuthology.orchestra.run.smithi161.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-12-16T21:39:04.305 INFO:teuthology.orchestra.run.smithi046.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2023-12-16T21:39:04.305 INFO:teuthology.orchestra.run.smithi161.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-12-16T21:39:04.305 INFO:teuthology.orchestra.run.smithi046.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-12-16T21:39:04.306 INFO:teuthology.orchestra.run.smithi161.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-12-16T21:39:04.310 INFO:teuthology.orchestra.run.smithi161.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-202-gc7586e95-1jammy [64.0 kB] 2023-12-16T21:39:04.313 INFO:teuthology.orchestra.run.smithi161.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-202-gc7586e95-1jammy [330 kB] 2023-12-16T21:39:04.316 INFO:teuthology.orchestra.run.smithi046.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-12-16T21:39:04.316 INFO:teuthology.orchestra.run.smithi046.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-12-16T21:39:04.316 INFO:teuthology.orchestra.run.smithi046.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-12-16T21:39:04.321 INFO:teuthology.orchestra.run.smithi161.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-12-16T21:39:04.325 INFO:teuthology.orchestra.run.smithi161.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-202-gc7586e95-1jammy [6,213 kB] 2023-12-16T21:39:04.336 INFO:teuthology.orchestra.run.smithi046.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-12-16T21:39:04.340 INFO:teuthology.orchestra.run.smithi046.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-12-16T21:39:04.342 INFO:teuthology.orchestra.run.smithi046.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-12-16T21:39:04.348 INFO:teuthology.orchestra.run.smithi046.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-12-16T21:39:04.348 INFO:teuthology.orchestra.run.smithi046.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-12-16T21:39:04.348 INFO:teuthology.orchestra.run.smithi046.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-12-16T21:39:04.350 INFO:teuthology.orchestra.run.smithi046.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-12-16T21:39:04.355 INFO:teuthology.orchestra.run.smithi046.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-12-16T21:39:04.569 INFO:teuthology.orchestra.run.smithi046.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-202-gc7586e95-1jammy [3,109 kB] 2023-12-16T21:39:04.588 INFO:teuthology.orchestra.run.smithi161.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-202-gc7586e95-1jammy [112 kB] 2023-12-16T21:39:04.591 INFO:teuthology.orchestra.run.smithi161.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-202-gc7586e95-1jammy [439 kB] 2023-12-16T21:39:04.607 INFO:teuthology.orchestra.run.smithi161.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-202-gc7586e95-1jammy [22.7 MB] 2023-12-16T21:39:04.975 INFO:teuthology.orchestra.run.smithi046.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-202-gc7586e95-1jammy [690 kB] 2023-12-16T21:39:05.076 INFO:teuthology.orchestra.run.smithi046.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-202-gc7586e95-1jammy [356 kB] 2023-12-16T21:39:05.129 INFO:teuthology.orchestra.run.smithi046.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-202-gc7586e95-1jammy [32.8 kB] 2023-12-16T21:39:05.133 INFO:teuthology.orchestra.run.smithi046.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-202-gc7586e95-1jammy [180 kB] 2023-12-16T21:39:05.155 INFO:teuthology.orchestra.run.smithi046.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-202-gc7586e95-1jammy [64.0 kB] 2023-12-16T21:39:05.163 INFO:teuthology.orchestra.run.smithi046.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-202-gc7586e95-1jammy [330 kB] 2023-12-16T21:39:05.204 INFO:teuthology.orchestra.run.smithi046.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-202-gc7586e95-1jammy [6,213 kB] 2023-12-16T21:39:05.738 INFO:teuthology.orchestra.run.smithi161.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-202-gc7586e95-1jammy [6,899 kB] 2023-12-16T21:39:05.799 INFO:teuthology.orchestra.run.smithi046.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-202-gc7586e95-1jammy [112 kB] 2023-12-16T21:39:05.802 INFO:teuthology.orchestra.run.smithi046.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-202-gc7586e95-1jammy [439 kB] 2023-12-16T21:39:05.846 INFO:teuthology.orchestra.run.smithi046.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-202-gc7586e95-1jammy [22.7 MB] 2023-12-16T21:39:06.027 INFO:teuthology.orchestra.run.smithi161.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-202-gc7586e95-1jammy [238 kB] 2023-12-16T21:39:06.041 INFO:teuthology.orchestra.run.smithi161.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-202-gc7586e95-1jammy [124 kB] 2023-12-16T21:39:06.047 INFO:teuthology.orchestra.run.smithi161.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-202-gc7586e95-1jammy [1,493 kB] 2023-12-16T21:39:06.141 INFO:teuthology.orchestra.run.smithi161.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-202-gc7586e95-1jammy [9,959 kB] 2023-12-16T21:39:06.905 INFO:teuthology.orchestra.run.smithi161.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-202-gc7586e95-1jammy [31.9 MB] 2023-12-16T21:39:07.341 INFO:teuthology.orchestra.run.smithi046.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-202-gc7586e95-1jammy [6,899 kB] 2023-12-16T21:39:07.663 INFO:teuthology.orchestra.run.smithi046.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-202-gc7586e95-1jammy [238 kB] 2023-12-16T21:39:07.675 INFO:teuthology.orchestra.run.smithi046.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-202-gc7586e95-1jammy [124 kB] 2023-12-16T21:39:07.682 INFO:teuthology.orchestra.run.smithi046.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-202-gc7586e95-1jammy [1,493 kB] 2023-12-16T21:39:07.769 INFO:teuthology.orchestra.run.smithi046.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-202-gc7586e95-1jammy [9,959 kB] 2023-12-16T21:39:08.345 INFO:teuthology.orchestra.run.smithi046.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-202-gc7586e95-1jammy [31.9 MB] 2023-12-16T21:39:09.474 INFO:teuthology.orchestra.run.smithi161.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-202-gc7586e95-1jammy [14.1 kB] 2023-12-16T21:39:09.474 INFO:teuthology.orchestra.run.smithi161.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-202-gc7586e95-1jammy [839 kB] 2023-12-16T21:39:09.491 INFO:teuthology.orchestra.run.smithi161.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-202-gc7586e95-1jammy [2,145 kB] 2023-12-16T21:39:09.551 INFO:teuthology.orchestra.run.smithi161.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-202-gc7586e95-1jammy [833 kB] 2023-12-16T21:39:09.621 INFO:teuthology.orchestra.run.smithi161.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-202-gc7586e95-1jammy [139 kB] 2023-12-16T21:39:09.628 INFO:teuthology.orchestra.run.smithi161.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-202-gc7586e95-1jammy [1,905 kB] 2023-12-16T21:39:09.782 INFO:teuthology.orchestra.run.smithi161.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-202-gc7586e95-1jammy [8,625 kB] 2023-12-16T21:39:10.217 INFO:teuthology.orchestra.run.smithi046.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-202-gc7586e95-1jammy [14.1 kB] 2023-12-16T21:39:10.218 INFO:teuthology.orchestra.run.smithi046.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-202-gc7586e95-1jammy [839 kB] 2023-12-16T21:39:10.236 INFO:teuthology.orchestra.run.smithi046.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-202-gc7586e95-1jammy [2,145 kB] 2023-12-16T21:39:10.287 INFO:teuthology.orchestra.run.smithi046.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-202-gc7586e95-1jammy [833 kB] 2023-12-16T21:39:10.332 INFO:teuthology.orchestra.run.smithi046.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-202-gc7586e95-1jammy [139 kB] 2023-12-16T21:39:10.341 INFO:teuthology.orchestra.run.smithi046.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-202-gc7586e95-1jammy [1,905 kB] 2023-12-16T21:39:10.446 INFO:teuthology.orchestra.run.smithi046.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-202-gc7586e95-1jammy [8,625 kB] 2023-12-16T21:39:10.464 INFO:teuthology.orchestra.run.smithi161.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-202-gc7586e95-1jammy [14.2 kB] 2023-12-16T21:39:10.465 INFO:teuthology.orchestra.run.smithi161.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-202-gc7586e95-1jammy [48.6 MB] 2023-12-16T21:39:10.922 INFO:teuthology.orchestra.run.smithi046.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-202-gc7586e95-1jammy [14.2 kB] 2023-12-16T21:39:10.922 INFO:teuthology.orchestra.run.smithi046.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-202-gc7586e95-1jammy [48.6 MB] 2023-12-16T21:39:13.782 INFO:teuthology.orchestra.run.smithi046.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-202-gc7586e95-1jammy [121 kB] 2023-12-16T21:39:13.785 INFO:teuthology.orchestra.run.smithi046.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-202-gc7586e95-1jammy [225 kB] 2023-12-16T21:39:13.790 INFO:teuthology.orchestra.run.smithi046.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-202-gc7586e95-1jammy [34.0 kB] 2023-12-16T21:39:13.791 INFO:teuthology.orchestra.run.smithi046.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-202-gc7586e95-1jammy [23.4 kB] 2023-12-16T21:39:13.792 INFO:teuthology.orchestra.run.smithi046.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-202-gc7586e95-1jammy [38.3 kB] 2023-12-16T21:39:13.793 INFO:teuthology.orchestra.run.smithi046.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-202-gc7586e95-1jammy [12.3 MB] 2023-12-16T21:39:14.357 INFO:teuthology.orchestra.run.smithi046.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-202-gc7586e95-1jammy [91.4 kB] 2023-12-16T21:39:14.594 INFO:teuthology.orchestra.run.smithi161.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-202-gc7586e95-1jammy [121 kB] 2023-12-16T21:39:14.597 INFO:teuthology.orchestra.run.smithi161.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-202-gc7586e95-1jammy [225 kB] 2023-12-16T21:39:14.601 INFO:teuthology.orchestra.run.smithi161.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-202-gc7586e95-1jammy [34.0 kB] 2023-12-16T21:39:14.602 INFO:teuthology.orchestra.run.smithi161.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-202-gc7586e95-1jammy [23.4 kB] 2023-12-16T21:39:14.603 INFO:teuthology.orchestra.run.smithi161.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-202-gc7586e95-1jammy [38.3 kB] 2023-12-16T21:39:14.604 INFO:teuthology.orchestra.run.smithi161.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-202-gc7586e95-1jammy [12.3 MB] 2023-12-16T21:39:14.916 INFO:teuthology.orchestra.run.smithi161.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-yuri8-testing-2023-12-15-0911/c7586e95519b7be76f3b6e3a4b333888baf46bc9/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-202-gc7586e95-1jammy [91.4 kB] 2023-12-16T21:39:14.997 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 187 MB in 11s (17.6 MB/s) 2023-12-16T21:39:15.134 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyperclip. 2023-12-16T21:39:15.235 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165989 files and directories currently installed.) 2023-12-16T21:39:15.241 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-12-16T21:39:15.282 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-12-16T21:39:15.519 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-wcwidth. 2023-12-16T21:39:15.521 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 187 MB in 11s (16.8 MB/s) 2023-12-16T21:39:15.546 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-12-16T21:39:15.577 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-16T21:39:15.642 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyperclip. 2023-12-16T21:39:15.727 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165989 files and directories currently installed.) 2023-12-16T21:39:15.733 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-12-16T21:39:15.774 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-12-16T21:39:15.905 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cmd2. 2023-12-16T21:39:15.932 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-12-16T21:39:15.971 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-12-16T21:39:16.077 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-wcwidth. 2023-12-16T21:39:16.103 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-12-16T21:39:16.143 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-16T21:39:16.358 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-12-16T21:39:16.384 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-12-16T21:39:16.466 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-16T21:39:16.697 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cmd2. 2023-12-16T21:39:16.723 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-12-16T21:39:16.755 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-12-16T21:39:16.812 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-12-16T21:39:16.838 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-12-16T21:39:16.878 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-16T21:39:17.066 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-12-16T21:39:17.093 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-12-16T21:39:17.157 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-16T21:39:17.164 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-12-16T21:39:17.191 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-12-16T21:39:17.231 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-16T21:39:17.486 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-12-16T21:39:17.512 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-12-16T21:39:17.552 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-16T21:39:17.676 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-12-16T21:39:17.703 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-16T21:39:17.751 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:39:17.889 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-12-16T21:39:17.915 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-12-16T21:39:17.955 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-16T21:39:18.162 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-12-16T21:39:18.190 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-16T21:39:18.229 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:39:18.367 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-12-16T21:39:18.393 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-16T21:39:18.442 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:39:18.574 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-12-16T21:39:18.601 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-16T21:39:18.640 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:39:18.794 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-12-16T21:39:18.810 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-16T21:39:18.853 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:39:18.994 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-12-16T21:39:19.021 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-12-16T21:39:19.060 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-16T21:39:19.181 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-12-16T21:39:19.208 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-16T21:39:19.247 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:39:19.321 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librados2. 2023-12-16T21:39:19.338 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../010-librados2_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:19.380 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librados2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:19.609 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-12-16T21:39:19.635 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-12-16T21:39:19.675 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-16T21:39:19.833 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librbd1. 2023-12-16T21:39:19.860 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../011-librbd1_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:19.899 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librbd1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:19.969 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librados2. 2023-12-16T21:39:19.985 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../010-librados2_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:20.019 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librados2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:20.335 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs2. 2023-12-16T21:39:20.363 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:20.402 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:20.439 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librbd1. 2023-12-16T21:39:20.466 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../011-librbd1_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:20.505 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librbd1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:20.663 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rados. 2023-12-16T21:39:20.690 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../013-python3-rados_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:20.721 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rados (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:20.942 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs2. 2023-12-16T21:39:20.968 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:21.008 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:21.049 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-12-16T21:39:21.076 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:21.115 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-argparse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:21.277 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rados. 2023-12-16T21:39:21.293 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../013-python3-rados_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:21.335 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rados (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:21.368 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cephfs. 2023-12-16T21:39:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:21.426 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cephfs (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:21.747 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-ceph-common. 2023-12-16T21:39:21.774 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:21.812 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:21.874 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-12-16T21:39:21.903 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:22.113 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-argparse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:22.116 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-prettytable. 2023-12-16T21:39:22.143 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-12-16T21:39:22.174 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-12-16T21:39:22.407 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cephfs. 2023-12-16T21:39:22.422 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:22.466 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cephfs (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:22.541 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rbd. 2023-12-16T21:39:22.567 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:22.702 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rbd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:22.726 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-ceph-common. 2023-12-16T21:39:22.742 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:22.776 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:23.055 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-12-16T21:39:23.062 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-prettytable. 2023-12-16T21:39:23.082 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-12-16T21:39:23.089 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2023-12-16T21:39:23.121 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-16T21:39:23.146 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-12-16T21:39:23.374 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rbd. 2023-12-16T21:39:23.376 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libreadline-dev:amd64. 2023-12-16T21:39:23.392 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2023-12-16T21:39:23.400 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:23.425 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2023-12-16T21:39:23.440 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rbd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:23.720 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2023-12-16T21:39:23.736 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2023-12-16T21:39:23.776 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-12-16T21:39:23.778 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-12-16T21:39:23.802 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-12-16T21:39:23.842 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-16T21:39:24.120 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libreadline-dev:amd64. 2023-12-16T21:39:24.132 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua5.1. 2023-12-16T21:39:24.135 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2023-12-16T21:39:24.159 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2023-12-16T21:39:24.178 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2023-12-16T21:39:24.198 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2023-12-16T21:39:24.481 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2023-12-16T21:39:24.496 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2023-12-16T21:39:24.526 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-any. 2023-12-16T21:39:24.531 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-12-16T21:39:24.542 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2023-12-16T21:39:24.576 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-any (27ubuntu1) ... 2023-12-16T21:39:24.796 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package zip. 2023-12-16T21:39:24.822 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2023-12-16T21:39:24.834 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package lua5.1. 2023-12-16T21:39:24.860 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2023-12-16T21:39:24.862 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking zip (3.0-12build2) ... 2023-12-16T21:39:24.900 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2023-12-16T21:39:25.203 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package lua-any. 2023-12-16T21:39:25.215 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package unzip. 2023-12-16T21:39:25.219 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2023-12-16T21:39:25.242 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2023-12-16T21:39:25.262 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking lua-any (27ubuntu1) ... 2023-12-16T21:39:25.282 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2023-12-16T21:39:25.489 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package zip. 2023-12-16T21:39:25.505 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2023-12-16T21:39:25.548 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking zip (3.0-12build2) ... 2023-12-16T21:39:25.695 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package luarocks. 2023-12-16T21:39:25.721 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2023-12-16T21:39:25.761 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2023-12-16T21:39:25.859 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package unzip. 2023-12-16T21:39:25.885 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.1_amd64.deb ... 2023-12-16T21:39:25.925 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking unzip (6.0-26ubuntu3.1) ... 2023-12-16T21:39:26.144 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package librgw2. 2023-12-16T21:39:26.171 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../027-librgw2_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:26.210 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking librgw2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:26.288 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package luarocks. 2023-12-16T21:39:26.314 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2023-12-16T21:39:26.346 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2023-12-16T21:39:26.703 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package librgw2. 2023-12-16T21:39:26.730 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../027-librgw2_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:26.746 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rgw. 2023-12-16T21:39:26.762 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:26.769 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking librgw2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:26.804 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:27.124 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liboath0:amd64. 2023-12-16T21:39:27.151 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2023-12-16T21:39:27.190 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-12-16T21:39:27.485 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libradosstriper1. 2023-12-16T21:39:27.489 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rgw. 2023-12-16T21:39:27.512 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:27.515 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:27.552 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libradosstriper1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:27.555 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:27.875 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liboath0:amd64. 2023-12-16T21:39:27.901 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2023-12-16T21:39:27.941 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-12-16T21:39:27.989 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-common. 2023-12-16T21:39:28.015 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../031-ceph-common_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:28.055 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:28.211 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libradosstriper1. 2023-12-16T21:39:28.237 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:28.269 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libradosstriper1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:28.689 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-common. 2023-12-16T21:39:28.716 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../031-ceph-common_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:28.755 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:29.319 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-base. 2023-12-16T21:39:29.345 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../032-ceph-base_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:29.395 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-base (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:29.919 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-base. 2023-12-16T21:39:29.945 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../032-ceph-base_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:29.986 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-base (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:30.097 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-12-16T21:39:30.124 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-12-16T21:39:30.164 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-12-16T21:39:30.450 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cheroot. 2023-12-16T21:39:30.477 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-12-16T21:39:30.526 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-16T21:39:30.706 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-12-16T21:39:30.732 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-12-16T21:39:30.794 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-12-16T21:39:30.845 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-12-16T21:39:30.872 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-12-16T21:39:30.912 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-12-16T21:39:31.100 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cheroot. 2023-12-16T21:39:31.127 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-12-16T21:39:31.167 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-16T21:39:31.215 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.text. 2023-12-16T21:39:31.242 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2023-12-16T21:39:31.281 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-12-16T21:39:31.461 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-12-16T21:39:31.487 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-12-16T21:39:31.527 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-12-16T21:39:31.534 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-12-16T21:39:31.561 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-12-16T21:39:31.601 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-12-16T21:39:31.797 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.text. 2023-12-16T21:39:31.823 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2023-12-16T21:39:31.864 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-12-16T21:39:31.887 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempora. 2023-12-16T21:39:31.914 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2023-12-16T21:39:31.953 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-12-16T21:39:32.150 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-12-16T21:39:32.177 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-12-16T21:39:32.208 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-12-16T21:39:32.265 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-portend. 2023-12-16T21:39:32.292 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2023-12-16T21:39:32.331 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-12-16T21:39:32.469 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tempora. 2023-12-16T21:39:32.496 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2023-12-16T21:39:32.536 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-12-16T21:39:32.601 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-12-16T21:39:32.628 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2023-12-16T21:39:32.659 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-12-16T21:39:32.863 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-portend. 2023-12-16T21:39:32.890 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2023-12-16T21:39:32.930 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-12-16T21:39:33.021 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cherrypy3. 2023-12-16T21:39:33.047 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2023-12-16T21:39:33.087 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-12-16T21:39:33.233 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-12-16T21:39:33.260 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2023-12-16T21:39:33.299 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-12-16T21:39:33.441 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-natsort. 2023-12-16T21:39:33.467 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2023-12-16T21:39:33.499 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-12-16T21:39:33.560 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cherrypy3. 2023-12-16T21:39:33.587 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2023-12-16T21:39:33.620 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-12-16T21:39:33.794 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-logutils. 2023-12-16T21:39:33.821 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2023-12-16T21:39:33.860 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-12-16T21:39:33.930 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-natsort. 2023-12-16T21:39:33.957 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2023-12-16T21:39:33.997 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-12-16T21:39:34.163 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-markupsafe. 2023-12-16T21:39:34.190 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-12-16T21:39:34.229 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-12-16T21:39:34.291 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-logutils. 2023-12-16T21:39:34.318 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2023-12-16T21:39:34.358 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-12-16T21:39:34.532 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-mako. 2023-12-16T21:39:34.559 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-12-16T21:39:34.599 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-16T21:39:34.660 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-markupsafe. 2023-12-16T21:39:34.687 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-12-16T21:39:34.727 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-12-16T21:39:34.886 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplegeneric. 2023-12-16T21:39:34.902 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2023-12-16T21:39:34.935 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-12-16T21:39:35.005 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-mako. 2023-12-16T21:39:35.031 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-12-16T21:39:35.063 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-16T21:39:35.189 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-singledispatch. 2023-12-16T21:39:35.215 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-12-16T21:39:35.255 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-12-16T21:39:35.375 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplegeneric. 2023-12-16T21:39:35.401 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2023-12-16T21:39:35.441 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-12-16T21:39:35.549 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webob. 2023-12-16T21:39:35.576 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-12-16T21:39:35.616 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-12-16T21:39:35.744 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-singledispatch. 2023-12-16T21:39:35.770 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-12-16T21:39:35.810 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-12-16T21:39:35.986 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-waitress. 2023-12-16T21:39:36.013 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-12-16T21:39:36.056 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-16T21:39:36.088 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webob. 2023-12-16T21:39:36.103 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-12-16T21:39:36.138 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-12-16T21:39:36.389 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-tempita. 2023-12-16T21:39:36.416 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-12-16T21:39:36.424 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-waitress. 2023-12-16T21:39:36.451 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-12-16T21:39:36.455 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-12-16T21:39:36.486 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-16T21:39:36.716 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-paste. 2023-12-16T21:39:36.732 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-12-16T21:39:36.743 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-tempita. 2023-12-16T21:39:36.766 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-12-16T21:39:36.770 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-12-16T21:39:36.810 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-12-16T21:39:37.094 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-12-16T21:39:37.095 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-paste. 2023-12-16T21:39:37.110 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-12-16T21:39:37.123 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-12-16T21:39:37.152 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-12-16T21:39:37.162 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-12-16T21:39:38.261 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-12-16T21:39:38.284 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-12-16T21:39:38.324 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-12-16T21:39:38.331 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastedeploy. 2023-12-16T21:39:38.358 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2023-12-16T21:39:38.398 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-12-16T21:39:38.651 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastedeploy. 2023-12-16T21:39:38.678 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2023-12-16T21:39:38.684 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-webtest. 2023-12-16T21:39:38.711 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2023-12-16T21:39:38.718 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-12-16T21:39:38.750 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-12-16T21:39:39.029 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-webtest. 2023-12-16T21:39:39.056 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2023-12-16T21:39:39.062 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pecan. 2023-12-16T21:39:39.089 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-12-16T21:39:39.095 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-12-16T21:39:39.128 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-12-16T21:39:39.407 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pecan. 2023-12-16T21:39:39.433 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-12-16T21:39:39.473 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-12-16T21:39:39.516 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-certifi. 2023-12-16T21:39:39.543 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2023-12-16T21:39:39.583 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-12-16T21:39:39.812 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-certifi. 2023-12-16T21:39:39.838 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2023-12-16T21:39:39.878 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-idna. 2023-12-16T21:39:39.878 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-12-16T21:39:39.905 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2023-12-16T21:39:39.936 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-idna (3.3-1) ... 2023-12-16T21:39:40.181 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-idna. 2023-12-16T21:39:40.207 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2023-12-16T21:39:40.230 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-urllib3. 2023-12-16T21:39:40.239 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-idna (3.3-1) ... 2023-12-16T21:39:40.257 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-12-16T21:39:40.297 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-16T21:39:40.500 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-urllib3. 2023-12-16T21:39:40.526 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-12-16T21:39:40.566 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-16T21:39:40.617 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests. 2023-12-16T21:39:40.644 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-12-16T21:39:40.684 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-16T21:39:40.878 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-requests. 2023-12-16T21:39:40.904 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-12-16T21:39:40.944 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-16T21:39:41.003 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-werkzeug. 2023-12-16T21:39:41.030 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-12-16T21:39:41.069 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-16T21:39:41.255 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-werkzeug. 2023-12-16T21:39:41.282 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-12-16T21:39:41.330 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-16T21:39:41.364 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-12-16T21:39:41.391 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:41.430 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-modules-core (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:41.625 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-12-16T21:39:41.640 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:41.683 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-modules-core (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:41.803 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-12-16T21:39:41.830 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:41.861 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:42.054 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-12-16T21:39:42.081 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:42.121 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:42.155 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-bcrypt. 2023-12-16T21:39:42.182 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-12-16T21:39:42.221 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-12-16T21:39:42.457 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-bcrypt. 2023-12-16T21:39:42.483 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-12-16T21:39:42.523 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-12-16T21:39:42.549 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-openssl. 2023-12-16T21:39:42.577 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2023-12-16T21:39:42.615 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-12-16T21:39:42.944 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr. 2023-12-16T21:39:42.967 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-openssl. 2023-12-16T21:39:42.971 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:42.994 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2023-12-16T21:39:43.010 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:43.034 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-12-16T21:39:43.337 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr. 2023-12-16T21:39:43.355 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mon. 2023-12-16T21:39:43.364 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:43.372 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:43.404 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:43.405 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mon (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:43.790 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mon. 2023-12-16T21:39:43.817 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:43.857 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mon (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:44.342 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-osd. 2023-12-16T21:39:44.369 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:44.409 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-osd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:44.760 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-osd. 2023-12-16T21:39:44.787 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:44.827 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-osd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:46.328 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph. 2023-12-16T21:39:46.345 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../068-ceph_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:46.386 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:46.613 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph. 2023-12-16T21:39:46.639 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../068-ceph_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:46.664 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-fuse. 2023-12-16T21:39:46.671 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:46.692 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:46.722 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:46.982 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-fuse. 2023-12-16T21:39:47.009 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:47.048 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:47.075 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mds. 2023-12-16T21:39:47.103 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:47.141 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mds (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:47.418 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mds. 2023-12-16T21:39:47.445 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:47.485 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mds (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:47.536 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephadm. 2023-12-16T21:39:47.564 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../071-cephadm_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:47.603 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:47.879 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephadm. 2023-12-16T21:39:47.906 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../071-cephadm_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:39:47.923 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-asyncssh. 2023-12-16T21:39:47.945 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:47.950 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2023-12-16T21:39:47.989 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-12-16T21:39:48.248 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-asyncssh. 2023-12-16T21:39:48.275 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2023-12-16T21:39:48.306 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-12-16T21:39:48.325 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-babel-localedata. 2023-12-16T21:39:48.353 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-12-16T21:39:48.391 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-16T21:39:48.609 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-babel-localedata. 2023-12-16T21:39:48.624 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-12-16T21:39:48.658 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-16T21:39:49.397 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-babel. 2023-12-16T21:39:49.413 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-12-16T21:39:49.446 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-12-16T21:39:49.597 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-babel. 2023-12-16T21:39:49.624 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-12-16T21:39:49.663 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-12-16T21:39:49.724 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-jinja2. 2023-12-16T21:39:49.740 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2023-12-16T21:39:49.774 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-12-16T21:39:50.008 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-jinja2. 2023-12-16T21:39:50.035 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1_all.deb ... 2023-12-16T21:39:50.074 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-12-16T21:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-12-16T21:39:50.105 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:50.136 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:50.414 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-repoze.lru. 2023-12-16T21:39:50.435 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-12-16T21:39:50.441 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2023-12-16T21:39:50.463 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:50.480 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-12-16T21:39:50.502 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:50.783 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-routes. 2023-12-16T21:39:50.810 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-12-16T21:39:50.850 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-12-16T21:39:50.865 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-repoze.lru. 2023-12-16T21:39:50.891 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2023-12-16T21:39:50.930 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-12-16T21:39:51.185 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-routes. 2023-12-16T21:39:51.195 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-12-16T21:39:51.211 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-12-16T21:39:51.222 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:51.243 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-12-16T21:39:51.261 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-dashboard (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:51.573 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-12-16T21:39:51.598 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:51.637 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-dashboard (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:51.723 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-12-16T21:39:51.751 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-12-16T21:39:51.789 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-16T21:39:52.057 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-12-16T21:39:52.084 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-12-16T21:39:52.124 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-16T21:39:52.318 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-joblib. 2023-12-16T21:39:52.346 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-12-16T21:39:52.384 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-12-16T21:39:52.586 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-joblib. 2023-12-16T21:39:52.613 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-12-16T21:39:52.644 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-12-16T21:39:52.696 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-12-16T21:39:52.724 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-12-16T21:39:52.848 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-12-16T21:39:52.997 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-12-16T21:39:53.024 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-12-16T21:39:53.153 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-sklearn. 2023-12-16T21:39:53.153 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-12-16T21:39:53.179 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-12-16T21:39:53.218 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-16T21:39:53.602 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-sklearn. 2023-12-16T21:39:53.630 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-12-16T21:39:53.661 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-16T21:39:53.969 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-12-16T21:39:53.997 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:54.035 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:54.370 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-12-16T21:39:54.397 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:54.436 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:54.980 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-cachetools. 2023-12-16T21:39:55.007 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2023-12-16T21:39:55.049 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-12-16T21:39:55.333 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1. 2023-12-16T21:39:55.339 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-cachetools. 2023-12-16T21:39:55.360 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2023-12-16T21:39:55.366 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2023-12-16T21:39:55.399 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-12-16T21:39:55.405 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-12-16T21:39:55.684 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1. 2023-12-16T21:39:55.711 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2023-12-16T21:39:55.751 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-12-16T21:39:55.811 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-12-16T21:39:55.838 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-12-16T21:39:55.886 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-12-16T21:39:56.070 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-12-16T21:39:56.097 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-12-16T21:39:56.136 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-12-16T21:39:56.214 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-rsa. 2023-12-16T21:39:56.242 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2023-12-16T21:39:56.280 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-rsa (4.8-1) ... 2023-12-16T21:39:56.448 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-rsa. 2023-12-16T21:39:56.475 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2023-12-16T21:39:56.514 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-rsa (4.8-1) ... 2023-12-16T21:39:56.634 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-google-auth. 2023-12-16T21:39:56.662 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2023-12-16T21:39:56.701 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-12-16T21:39:56.835 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-google-auth. 2023-12-16T21:39:56.863 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2023-12-16T21:39:56.901 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-12-16T21:39:57.012 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-12-16T21:39:57.040 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-12-16T21:39:57.079 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-16T21:39:57.180 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-12-16T21:39:57.207 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-12-16T21:39:57.238 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-16T21:39:57.348 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-websocket. 2023-12-16T21:39:57.365 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2023-12-16T21:39:57.398 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-12-16T21:39:57.516 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-websocket. 2023-12-16T21:39:57.543 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2023-12-16T21:39:57.582 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-12-16T21:39:57.702 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-kubernetes. 2023-12-16T21:39:57.729 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-12-16T21:39:57.768 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-16T21:39:57.936 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-kubernetes. 2023-12-16T21:39:57.963 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-12-16T21:39:58.002 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-16T21:39:58.474 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-12-16T21:39:58.502 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:58.557 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:58.759 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-12-16T21:39:58.785 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:39:58.883 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:39:58.968 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libonig5:amd64. 2023-12-16T21:39:58.996 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-12-16T21:39:59.035 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-16T21:39:59.186 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libonig5:amd64. 2023-12-16T21:39:59.213 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-12-16T21:39:59.252 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-16T21:39:59.380 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjq1:amd64. 2023-12-16T21:39:59.408 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-12-16T21:39:59.446 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-16T21:39:59.580 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjq1:amd64. 2023-12-16T21:39:59.607 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-12-16T21:39:59.663 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-16T21:39:59.699 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package jq. 2023-12-16T21:39:59.715 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-12-16T21:39:59.748 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-12-16T21:39:59.916 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package jq. 2023-12-16T21:39:59.943 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-12-16T21:39:59.969 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package socat. 2023-12-16T21:39:59.982 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-12-16T21:39:59.996 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-12-16T21:40:00.034 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-12-16T21:40:00.177 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package socat. 2023-12-16T21:40:00.192 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-12-16T21:40:00.235 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-12-16T21:40:00.354 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package xmlstarlet. 2023-12-16T21:40:00.371 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-12-16T21:40:00.413 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-12-16T21:40:00.505 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package xmlstarlet. 2023-12-16T21:40:00.521 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-12-16T21:40:00.563 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-12-16T21:40:00.750 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-test. 2023-12-16T21:40:00.778 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../099-ceph-test_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:00.817 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-test (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:00.925 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-test. 2023-12-16T21:40:00.941 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../099-ceph-test_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:00.984 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-test (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:03.181 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ceph-volume. 2023-12-16T21:40:03.209 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:40:03.249 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ceph-volume (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:03.280 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ceph-volume. 2023-12-16T21:40:03.307 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:40:03.346 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ceph-volume (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:03.734 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-mirror. 2023-12-16T21:40:03.735 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-mirror. 2023-12-16T21:40:03.761 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:03.763 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:03.847 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-mirror (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:03.902 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-mirror (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:04.238 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-colorama. 2023-12-16T21:40:04.266 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2023-12-16T21:40:04.304 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-12-16T21:40:04.328 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-colorama. 2023-12-16T21:40:04.355 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2023-12-16T21:40:04.394 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-12-16T21:40:04.615 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-shell. 2023-12-16T21:40:04.644 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:40:04.655 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-shell. 2023-12-16T21:40:04.671 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:40:04.690 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-shell (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:04.713 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-shell (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:04.941 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package cephfs-top. 2023-12-16T21:40:04.969 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:40:05.001 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package cephfs-top. 2023-12-16T21:40:05.007 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking cephfs-top (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:05.017 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-202-gc7586e95-1jammy_all.deb ... 2023-12-16T21:40:05.067 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking cephfs-top (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:05.260 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libcephfs-dev. 2023-12-16T21:40:05.276 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:05.318 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libcephfs-dev (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:05.345 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libcephfs-dev. 2023-12-16T21:40:05.362 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:05.403 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libcephfs-dev (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:05.655 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package lua-socket:amd64. 2023-12-16T21:40:05.682 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2023-12-16T21:40:05.721 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-12-16T21:40:05.731 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-socket:amd64. 2023-12-16T21:40:05.760 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2023-12-16T21:40:05.806 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-12-16T21:40:06.051 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package lua-sec:amd64. 2023-12-16T21:40:06.078 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2023-12-16T21:40:06.117 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2023-12-16T21:40:06.118 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package lua-sec:amd64. 2023-12-16T21:40:06.146 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2023-12-16T21:40:06.185 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2023-12-16T21:40:06.463 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package nvme-cli. 2023-12-16T21:40:06.490 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-12-16T21:40:06.529 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-12-16T21:40:06.555 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package nvme-cli. 2023-12-16T21:40:06.584 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-12-16T21:40:06.630 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-12-16T21:40:06.934 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package pkg-config. 2023-12-16T21:40:06.962 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2023-12-16T21:40:07.001 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2023-12-16T21:40:07.060 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pkg-config. 2023-12-16T21:40:07.088 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2023-12-16T21:40:07.126 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2023-12-16T21:40:07.262 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-12-16T21:40:07.278 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-12-16T21:40:07.320 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-12-16T21:40:07.404 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-12-16T21:40:07.421 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-12-16T21:40:07.462 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-12-16T21:40:07.773 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-iniconfig. 2023-12-16T21:40:07.801 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2023-12-16T21:40:07.840 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-12-16T21:40:07.899 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-iniconfig. 2023-12-16T21:40:07.927 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2023-12-16T21:40:07.965 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-12-16T21:40:08.143 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pastescript. 2023-12-16T21:40:08.170 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2023-12-16T21:40:08.209 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-12-16T21:40:08.209 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pastescript. 2023-12-16T21:40:08.238 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2023-12-16T21:40:08.268 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-12-16T21:40:08.537 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pluggy. 2023-12-16T21:40:08.563 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pluggy. 2023-12-16T21:40:08.565 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2023-12-16T21:40:08.591 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2023-12-16T21:40:08.596 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-12-16T21:40:08.629 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-12-16T21:40:08.898 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-psutil. 2023-12-16T21:40:08.926 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-12-16T21:40:08.941 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-psutil. 2023-12-16T21:40:08.965 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-12-16T21:40:08.968 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-12-16T21:40:09.152 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-12-16T21:40:09.326 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-py. 2023-12-16T21:40:09.354 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2023-12-16T21:40:09.393 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-py (1.10.0-1) ... 2023-12-16T21:40:09.518 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-py. 2023-12-16T21:40:09.546 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2023-12-16T21:40:09.585 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-py (1.10.0-1) ... 2023-12-16T21:40:09.713 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pygments. 2023-12-16T21:40:09.740 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-12-16T21:40:09.779 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-12-16T21:40:09.905 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pygments. 2023-12-16T21:40:09.933 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-12-16T21:40:09.971 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-12-16T21:40:10.233 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pyinotify. 2023-12-16T21:40:10.261 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-12-16T21:40:10.299 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-12-16T21:40:10.492 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pyinotify. 2023-12-16T21:40:10.520 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-12-16T21:40:10.558 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-12-16T21:40:10.578 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-toml. 2023-12-16T21:40:10.605 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2023-12-16T21:40:10.644 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-12-16T21:40:10.852 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-toml. 2023-12-16T21:40:10.881 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2023-12-16T21:40:10.905 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-pytest. 2023-12-16T21:40:10.919 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-12-16T21:40:10.933 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-12-16T21:40:10.971 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-12-16T21:40:11.163 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-pytest. 2023-12-16T21:40:11.180 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-12-16T21:40:11.213 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-12-16T21:40:11.308 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package python3-simplejson. 2023-12-16T21:40:11.336 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-12-16T21:40:11.375 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-12-16T21:40:11.567 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package python3-simplejson. 2023-12-16T21:40:11.595 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-12-16T21:40:11.633 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-12-16T21:40:11.644 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-12-16T21:40:11.660 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-12-16T21:40:11.694 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-12-16T21:40:11.911 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-12-16T21:40:11.927 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-12-16T21:40:11.960 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-12-16T21:40:12.398 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package radosgw. 2023-12-16T21:40:12.426 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../122-radosgw_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:12.464 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking radosgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:12.673 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package radosgw. 2023-12-16T21:40:12.702 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../122-radosgw_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:12.748 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking radosgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:13.276 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package rbd-fuse. 2023-12-16T21:40:13.304 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:13.342 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking rbd-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:13.534 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package rbd-fuse. 2023-12-16T21:40:13.551 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-202-gc7586e95-1jammy_amd64.deb ... 2023-12-16T21:40:13.592 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking rbd-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:13.647 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-12-16T21:40:13.856 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-12-16T21:40:13.906 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-16T21:40:14.037 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-12-16T21:40:14.105 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-16T21:40:14.215 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-12-16T21:40:14.482 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-12-16T21:40:14.532 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-12-16T21:40:14.809 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-12-16T21:40:14.888 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-12-16T21:40:15.250 INFO:teuthology.orchestra.run.smithi046.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-12-16T21:40:15.317 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:15.349 INFO:teuthology.orchestra.run.smithi161.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-12-16T21:40:15.409 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:15.843 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user cephadm....done 2023-12-16T21:40:15.944 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user cephadm....done 2023-12-16T21:40:15.956 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-16T21:40:16.065 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-16T21:40:16.241 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-12-16T21:40:16.353 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-12-16T21:40:16.509 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-12-16T21:40:16.610 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-12-16T21:40:16.710 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-12-16T21:40:16.811 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-12-16T21:40:16.852 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-12-16T21:40:17.078 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-12-16T21:40:17.112 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-12-16T21:40:17.238 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-py (1.10.0-1) ... 2023-12-16T21:40:17.321 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-12-16T21:40:17.447 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-py (1.10.0-1) ... 2023-12-16T21:40:17.539 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-12-16T21:40:17.774 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-12-16T21:40:17.916 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-12-16T21:40:18.126 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-12-16T21:40:18.176 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-12-16T21:40:18.377 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-12-16T21:40:18.444 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-12-16T21:40:18.653 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-12-16T21:40:18.703 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2023-12-16T21:40:18.921 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up unzip (6.0-26ubuntu3.1) ... 2023-12-16T21:40:18.946 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-12-16T21:40:19.198 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-12-16T21:40:19.214 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-12-16T21:40:19.449 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-12-16T21:40:19.482 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-12-16T21:40:19.708 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-12-16T21:40:19.816 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-argparse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:19.976 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-argparse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:20.084 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-16T21:40:20.210 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-12-16T21:40:20.252 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-16T21:40:20.336 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-12-16T21:40:20.353 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2023-12-16T21:40:20.479 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-12-16T21:40:20.595 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2023-12-16T21:40:20.712 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-12-16T21:40:20.738 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2023-12-16T21:40:20.864 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-12-16T21:40:20.955 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2023-12-16T21:40:21.123 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2023-12-16T21:40:21.356 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2023-12-16T21:40:21.386 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2023-12-16T21:40:21.450 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-16T21:40:21.519 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2023-12-16T21:40:21.551 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-12-16T21:40:21.554 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2023-12-16T21:40:21.618 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-16T21:40:21.720 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-12-16T21:40:21.919 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-16T21:40:22.045 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-12-16T21:40:22.070 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-16T21:40:22.171 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-12-16T21:40:22.312 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-12-16T21:40:22.439 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-12-16T21:40:22.597 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-12-16T21:40:22.698 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-12-16T21:40:22.898 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up zip (3.0-12build2) ... 2023-12-16T21:40:23.008 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up zip (3.0-12build2) ... 2023-12-16T21:40:23.016 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-12-16T21:40:23.126 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-12-16T21:40:23.751 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-12-16T21:40:23.811 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-12-16T21:40:24.002 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-12-16T21:40:24.079 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-12-16T21:40:24.111 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-12-16T21:40:24.205 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-12-16T21:40:24.238 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-16T21:40:24.356 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-16T21:40:24.530 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-12-16T21:40:24.649 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-12-16T21:40:24.898 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-12-16T21:40:25.050 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-12-16T21:40:25.141 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-12-16T21:40:25.301 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-12-16T21:40:25.525 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-16T21:40:25.669 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-16T21:40:25.810 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-12-16T21:40:25.954 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-12-16T21:40:26.136 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-16T21:40:26.255 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-16T21:40:26.479 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-12-16T21:40:26.598 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-12-16T21:40:26.713 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-12-16T21:40:26.823 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:26.849 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-12-16T21:40:26.992 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:27.124 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-idna (3.3-1) ... 2023-12-16T21:40:27.302 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-idna (3.3-1) ... 2023-12-16T21:40:27.458 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-16T21:40:27.611 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-16T21:40:28.680 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2023-12-16T21:40:28.767 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2023-12-16T21:40:28.957 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:40:29.010 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:40:29.091 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-toml (0.10.2-1) ... 2023-12-16T21:40:29.119 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-toml (0.10.2-1) ... 2023-12-16T21:40:29.343 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-16T21:40:29.370 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-16T21:40:29.469 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-12-16T21:40:29.497 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-12-16T21:40:29.595 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-16T21:40:29.626 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-16T21:40:29.887 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-12-16T21:40:29.916 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-12-16T21:40:30.155 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-12-16T21:40:30.183 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-12-16T21:40:30.398 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:40:30.434 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:40:30.524 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-12-16T21:40:30.552 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-12-16T21:40:30.828 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-12-16T21:40:30.833 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-12-16T21:40:31.079 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-12-16T21:40:31.093 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-12-16T21:40:31.347 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-12-16T21:40:31.352 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-12-16T21:40:31.598 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-12-16T21:40:31.603 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-12-16T21:40:31.838 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-12-16T21:40:31.866 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-12-16T21:40:32.080 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-16T21:40:32.150 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-16T21:40:32.206 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-12-16T21:40:32.293 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-12-16T21:40:32.491 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-16T21:40:32.561 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-16T21:40:32.617 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-16T21:40:32.679 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-16T21:40:32.909 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-12-16T21:40:32.971 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-12-16T21:40:33.210 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-12-16T21:40:33.272 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-12-16T21:40:33.445 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-12-16T21:40:33.523 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2023-12-16T21:40:33.546 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2023-12-16T21:40:33.649 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2023-12-16T21:40:33.672 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-12-16T21:40:33.761 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-12-16T21:40:33.784 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-12-16T21:40:33.873 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-12-16T21:40:34.049 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-16T21:40:34.127 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-16T21:40:34.174 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-12-16T21:40:34.261 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-12-16T21:40:34.567 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-12-16T21:40:34.655 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-12-16T21:40:34.819 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up lua-any (27ubuntu1) ... 2023-12-16T21:40:34.939 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up lua-any (27ubuntu1) ... 2023-12-16T21:40:34.944 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-portend (3.0.0-1) ... 2023-12-16T21:40:35.065 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-portend (3.0.0-1) ... 2023-12-16T21:40:35.195 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:40:35.296 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-12-16T21:40:35.333 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-16T21:40:35.475 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-12-16T21:40:35.614 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-12-16T21:40:35.785 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-12-16T21:40:35.923 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-16T21:40:36.094 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-16T21:40:36.225 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-12-16T21:40:36.362 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-12-16T21:40:36.453 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rsa (4.8-1) ... 2023-12-16T21:40:36.565 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rsa (4.8-1) ... 2023-12-16T21:40:36.712 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-12-16T21:40:36.833 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-12-16T21:40:36.972 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-16T21:40:37.103 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-16T21:40:37.232 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-12-16T21:40:37.362 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-12-16T21:40:37.616 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-12-16T21:40:37.739 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-12-16T21:40:37.910 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-12-16T21:40:38.032 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-12-16T21:40:38.368 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-16T21:40:38.387 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-16T21:40:38.469 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librados2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:38.504 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librados2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:38.570 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libsqlite3-mod-ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:38.633 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libsqlite3-mod-ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:38.688 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2023-12-16T21:40:38.765 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2023-12-16T21:40:39.024 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.101 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.150 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libradosstriper1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.219 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libradosstriper1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.276 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-12-16T21:40:39.328 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-12-16T21:40:39.553 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librbd1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.588 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librbd1 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.679 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-modules-core (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.689 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-modules-core (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.798 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:39.805 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.015 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-16T21:40:40.016 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-16T21:40:40.023 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-16T21:40:40.023 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-16T21:40:40.473 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libcephfs-dev (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.516 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libcephfs-dev (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.599 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rados (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.642 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rados (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.725 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up librgw2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.768 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up librgw2 (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.851 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-top (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.894 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-top (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:40.960 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-16T21:40:41.045 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-16T21:40:42.139 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rbd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.265 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up rbd-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.349 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rbd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.391 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-rgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.474 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up rbd-fuse (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.517 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up python3-cephfs (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.600 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-rgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.643 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-shell (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.726 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up python3-cephfs (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.777 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.852 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-shell (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:42.978 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-common (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:43.303 INFO:teuthology.orchestra.run.smithi161.stdout:Adding group ceph....done 2023-12-16T21:40:43.429 INFO:teuthology.orchestra.run.smithi046.stdout:Adding group ceph....done 2023-12-16T21:40:43.679 INFO:teuthology.orchestra.run.smithi161.stdout:Adding system user ceph....done 2023-12-16T21:40:43.780 INFO:teuthology.orchestra.run.smithi046.stdout:Adding system user ceph....done 2023-12-16T21:40:43.831 INFO:teuthology.orchestra.run.smithi161.stdout:Setting system user ceph properties....done 2023-12-16T21:40:43.844 INFO:teuthology.orchestra.run.smithi161.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-12-16T21:40:43.915 INFO:teuthology.orchestra.run.smithi046.stdout:Setting system user ceph properties....done 2023-12-16T21:40:43.933 INFO:teuthology.orchestra.run.smithi046.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-12-16T21:40:43.982 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-12-16T21:40:44.104 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-12-16T21:40:44.290 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-12-16T21:40:44.415 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-12-16T21:40:44.804 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up cephfs-mirror (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:44.871 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up cephfs-mirror (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:45.072 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-16T21:40:45.072 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-16T21:40:45.207 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-16T21:40:45.207 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-16T21:40:45.547 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-test (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:45.682 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up radosgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:45.709 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-test (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:45.920 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up radosgw (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:46.178 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-16T21:40:46.178 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-16T21:40:46.523 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-16T21:40:46.523 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-16T21:40:46.650 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-base (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:46.984 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-base (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:47.004 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-12-16T21:40:47.331 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-12-16T21:40:47.469 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mds (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:47.710 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-16T21:40:47.711 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-16T21:40:47.786 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mds (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:48.015 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-16T21:40:48.015 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-16T21:40:48.170 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:48.380 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-16T21:40:48.380 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-16T21:40:48.446 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:48.659 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-16T21:40:48.659 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-16T21:40:48.855 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-osd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:49.114 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-osd (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:49.141 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-16T21:40:49.141 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-16T21:40:49.396 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-16T21:40:49.396 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-16T21:40:49.631 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-k8sevents (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:49.757 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:49.882 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-k8sevents (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:49.908 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mon (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:50.008 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:50.127 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-16T21:40:50.128 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-16T21:40:50.159 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mon (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:50.370 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-16T21:40:50.370 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-16T21:40:50.672 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:50.832 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:50.856 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-cephadm (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:50.958 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-mgr-dashboard (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:51.024 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:51.109 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ceph-volume (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:51.159 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-mgr-dashboard (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:51.310 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ceph-volume (19.0.0-202-gc7586e95-1jammy) ... 2023-12-16T21:40:51.495 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2023-12-16T21:40:51.595 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-16T21:40:51.730 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2023-12-16T21:40:51.748 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-16T21:40:52.073 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-16T21:40:52.209 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-16T21:40:52.917 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-16T21:40:52.990 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-16T21:40:53.540 INFO:teuthology.orchestra.run.smithi161.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-16T21:40:53.544 DEBUG:teuthology.orchestra.run.smithi161:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-12-16T21:40:53.599 INFO:teuthology.orchestra.run.smithi161.stdout:Reading package lists... 2023-12-16T21:40:53.657 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-16T21:40:53.659 DEBUG:teuthology.orchestra.run.smithi046:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-12-16T21:40:53.759 INFO:teuthology.orchestra.run.smithi046.stdout:Reading package lists... 2023-12-16T21:40:53.789 INFO:teuthology.orchestra.run.smithi161.stdout:Building dependency tree... 2023-12-16T21:40:53.790 INFO:teuthology.orchestra.run.smithi161.stdout:Reading state information... 2023-12-16T21:40:53.958 INFO:teuthology.orchestra.run.smithi161.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-12-16T21:40:53.958 INFO:teuthology.orchestra.run.smithi161.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-12-16T21:40:53.958 INFO:teuthology.orchestra.run.smithi161.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-12-16T21:40:53.959 INFO:teuthology.orchestra.run.smithi161.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-12-16T21:40:53.959 INFO:teuthology.orchestra.run.smithi161.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-12-16T21:40:53.959 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:40:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:40:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:40:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:40:53.959 INFO:teuthology.orchestra.run.smithi161.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-16T21:40:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-16T21:40:53.960 INFO:teuthology.orchestra.run.smithi161.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:40:53.960 INFO:teuthology.orchestra.run.smithi161.stdout:The following additional packages will be installed: 2023-12-16T21:40:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-12-16T21:40:53.960 INFO:teuthology.orchestra.run.smithi161.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-12-16T21:40:53.961 INFO:teuthology.orchestra.run.smithi046.stdout:Building dependency tree... 2023-12-16T21:40:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-12-16T21:40:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-12-16T21:40:53.961 INFO:teuthology.orchestra.run.smithi161.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-16T21:40:53.962 INFO:teuthology.orchestra.run.smithi046.stdout:Reading state information... 2023-12-16T21:40:53.962 INFO:teuthology.orchestra.run.smithi161.stdout:Suggested packages: 2023-12-16T21:40:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-12-16T21:40:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-12-16T21:40:53.962 INFO:teuthology.orchestra.run.smithi161.stdout: libengine-pkcs11-openssl 2023-12-16T21:40:53.999 INFO:teuthology.orchestra.run.smithi161.stdout:The following NEW packages will be installed: 2023-12-16T21:40:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-12-16T21:40:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-12-16T21:40:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-12-16T21:40:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-12-16T21:40:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-12-16T21:40:54.000 INFO:teuthology.orchestra.run.smithi161.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-12-16T21:40:54.001 INFO:teuthology.orchestra.run.smithi161.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-16T21:40:54.001 INFO:teuthology.orchestra.run.smithi161.stdout:The following packages will be upgraded: 2023-12-16T21:40:54.001 INFO:teuthology.orchestra.run.smithi161.stdout: libssl3 2023-12-16T21:40:54.050 INFO:teuthology.orchestra.run.smithi161.stdout:1 upgraded, 39 newly installed, 0 to remove and 264 not upgraded. 2023-12-16T21:40:54.050 INFO:teuthology.orchestra.run.smithi161.stdout:Need to get 54.6 MB of archives. 2023-12-16T21:40:54.050 INFO:teuthology.orchestra.run.smithi161.stdout:After this operation, 202 MB of additional disk space will be used. 2023-12-16T21:40:54.051 INFO:teuthology.orchestra.run.smithi161.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-12-16T21:40:54.140 INFO:teuthology.orchestra.run.smithi046.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-12-16T21:40:54.140 INFO:teuthology.orchestra.run.smithi046.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-12-16T21:40:54.140 INFO:teuthology.orchestra.run.smithi046.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-12-16T21:40:54.140 INFO:teuthology.orchestra.run.smithi046.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages were automatically installed and are no longer required: 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-16T21:40:54.141 INFO:teuthology.orchestra.run.smithi046.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-16T21:40:54.142 INFO:teuthology.orchestra.run.smithi046.stdout:The following additional packages will be installed: 2023-12-16T21:40:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-12-16T21:40:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-12-16T21:40:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-12-16T21:40:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-12-16T21:40:54.142 INFO:teuthology.orchestra.run.smithi046.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-16T21:40:54.143 INFO:teuthology.orchestra.run.smithi046.stdout:Suggested packages: 2023-12-16T21:40:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-12-16T21:40:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-12-16T21:40:54.143 INFO:teuthology.orchestra.run.smithi046.stdout: libengine-pkcs11-openssl 2023-12-16T21:40:54.150 INFO:teuthology.orchestra.run.smithi161.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-12-16T21:40:54.184 INFO:teuthology.orchestra.run.smithi046.stdout:The following NEW packages will be installed: 2023-12-16T21:40:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-12-16T21:40:54.185 INFO:teuthology.orchestra.run.smithi046.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-12-16T21:40:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-12-16T21:40:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-12-16T21:40:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-12-16T21:40:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-12-16T21:40:54.186 INFO:teuthology.orchestra.run.smithi046.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-16T21:40:54.187 INFO:teuthology.orchestra.run.smithi046.stdout:The following packages will be upgraded: 2023-12-16T21:40:54.187 INFO:teuthology.orchestra.run.smithi046.stdout: libssl3 2023-12-16T21:40:54.214 INFO:teuthology.orchestra.run.smithi161.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-12-16T21:40:54.227 INFO:teuthology.orchestra.run.smithi161.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-12-16T21:40:54.228 INFO:teuthology.orchestra.run.smithi161.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-12-16T21:40:54.230 INFO:teuthology.orchestra.run.smithi161.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-12-16T21:40:54.232 INFO:teuthology.orchestra.run.smithi161.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-12-16T21:40:54.232 INFO:teuthology.orchestra.run.smithi161.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2023-12-16T21:40:54.234 INFO:teuthology.orchestra.run.smithi161.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-12-16T21:40:54.235 INFO:teuthology.orchestra.run.smithi161.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-12-16T21:40:54.236 INFO:teuthology.orchestra.run.smithi161.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-12-16T21:40:54.236 INFO:teuthology.orchestra.run.smithi161.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-12-16T21:40:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:1 upgraded, 39 newly installed, 0 to remove and 264 not upgraded. 2023-12-16T21:40:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:Need to get 54.6 MB of archives. 2023-12-16T21:40:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:After this operation, 202 MB of additional disk space will be used. 2023-12-16T21:40:54.244 INFO:teuthology.orchestra.run.smithi046.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-12-16T21:40:54.252 INFO:teuthology.orchestra.run.smithi161.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-12-16T21:40:54.361 INFO:teuthology.orchestra.run.smithi046.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-12-16T21:40:54.430 INFO:teuthology.orchestra.run.smithi046.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-12-16T21:40:54.446 INFO:teuthology.orchestra.run.smithi046.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-12-16T21:40:54.447 INFO:teuthology.orchestra.run.smithi046.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-12-16T21:40:54.450 INFO:teuthology.orchestra.run.smithi046.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-12-16T21:40:54.451 INFO:teuthology.orchestra.run.smithi046.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-12-16T21:40:54.452 INFO:teuthology.orchestra.run.smithi046.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2023-12-16T21:40:54.454 INFO:teuthology.orchestra.run.smithi046.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-12-16T21:40:54.455 INFO:teuthology.orchestra.run.smithi046.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-12-16T21:40:54.456 INFO:teuthology.orchestra.run.smithi046.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-12-16T21:40:54.462 INFO:teuthology.orchestra.run.smithi046.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-12-16T21:40:54.476 INFO:teuthology.orchestra.run.smithi046.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-12-16T21:40:54.603 INFO:teuthology.orchestra.run.smithi161.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-12-16T21:40:54.605 INFO:teuthology.orchestra.run.smithi161.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-12-16T21:40:54.605 INFO:teuthology.orchestra.run.smithi161.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-12-16T21:40:54.608 INFO:teuthology.orchestra.run.smithi161.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-12-16T21:40:54.608 INFO:teuthology.orchestra.run.smithi161.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-12-16T21:40:54.613 INFO:teuthology.orchestra.run.smithi161.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2023-12-16T21:40:54.615 INFO:teuthology.orchestra.run.smithi161.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-12-16T21:40:54.624 INFO:teuthology.orchestra.run.smithi161.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-12-16T21:40:54.625 INFO:teuthology.orchestra.run.smithi161.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-12-16T21:40:54.650 INFO:teuthology.orchestra.run.smithi161.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-12-16T21:40:54.650 INFO:teuthology.orchestra.run.smithi161.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-12-16T21:40:54.661 INFO:teuthology.orchestra.run.smithi161.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-12-16T21:40:54.682 INFO:teuthology.orchestra.run.smithi161.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-12-16T21:40:54.684 INFO:teuthology.orchestra.run.smithi161.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-12-16T21:40:54.685 INFO:teuthology.orchestra.run.smithi161.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-12-16T21:40:54.685 INFO:teuthology.orchestra.run.smithi161.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-12-16T21:40:54.686 INFO:teuthology.orchestra.run.smithi161.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2023-12-16T21:40:54.696 INFO:teuthology.orchestra.run.smithi161.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-12-16T21:40:54.698 INFO:teuthology.orchestra.run.smithi161.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2023-12-16T21:40:54.894 INFO:teuthology.orchestra.run.smithi161.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-12-16T21:40:54.894 INFO:teuthology.orchestra.run.smithi161.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-12-16T21:40:54.894 INFO:teuthology.orchestra.run.smithi161.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-12-16T21:40:54.894 INFO:teuthology.orchestra.run.smithi161.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-12-16T21:40:54.894 INFO:teuthology.orchestra.run.smithi161.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-12-16T21:40:54.894 INFO:teuthology.orchestra.run.smithi161.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-12-16T21:40:54.906 INFO:teuthology.orchestra.run.smithi161.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-12-16T21:40:54.906 INFO:teuthology.orchestra.run.smithi161.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-12-16T21:40:54.979 INFO:teuthology.orchestra.run.smithi046.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-12-16T21:40:54.982 INFO:teuthology.orchestra.run.smithi046.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-12-16T21:40:54.983 INFO:teuthology.orchestra.run.smithi046.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-12-16T21:40:54.988 INFO:teuthology.orchestra.run.smithi046.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-12-16T21:40:54.988 INFO:teuthology.orchestra.run.smithi046.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-12-16T21:40:54.996 INFO:teuthology.orchestra.run.smithi046.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2023-12-16T21:40:54.999 INFO:teuthology.orchestra.run.smithi046.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-12-16T21:40:55.012 INFO:teuthology.orchestra.run.smithi046.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-12-16T21:40:55.012 INFO:teuthology.orchestra.run.smithi046.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-12-16T21:40:55.045 INFO:teuthology.orchestra.run.smithi046.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-12-16T21:40:55.046 INFO:teuthology.orchestra.run.smithi046.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-12-16T21:40:55.058 INFO:teuthology.orchestra.run.smithi046.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-12-16T21:40:55.082 INFO:teuthology.orchestra.run.smithi046.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-12-16T21:40:55.088 INFO:teuthology.orchestra.run.smithi046.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-12-16T21:40:55.089 INFO:teuthology.orchestra.run.smithi046.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-12-16T21:40:55.089 INFO:teuthology.orchestra.run.smithi046.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-12-16T21:40:55.090 INFO:teuthology.orchestra.run.smithi046.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2023-12-16T21:40:55.106 INFO:teuthology.orchestra.run.smithi046.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-12-16T21:40:55.109 INFO:teuthology.orchestra.run.smithi046.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2023-12-16T21:40:55.298 INFO:teuthology.orchestra.run.smithi161.stdout:Preconfiguring packages ... 2023-12-16T21:40:55.434 INFO:teuthology.orchestra.run.smithi046.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-12-16T21:40:55.434 INFO:teuthology.orchestra.run.smithi046.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-12-16T21:40:55.434 INFO:teuthology.orchestra.run.smithi046.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-12-16T21:40:55.435 INFO:teuthology.orchestra.run.smithi046.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-12-16T21:40:55.435 INFO:teuthology.orchestra.run.smithi046.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-12-16T21:40:55.435 INFO:teuthology.orchestra.run.smithi046.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-12-16T21:40:55.449 INFO:teuthology.orchestra.run.smithi161.stdout:Fetched 54.6 MB in 1s (60.7 MB/s) 2023-12-16T21:40:55.451 INFO:teuthology.orchestra.run.smithi046.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-12-16T21:40:55.452 INFO:teuthology.orchestra.run.smithi046.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-12-16T21:40:55.831 INFO:teuthology.orchestra.run.smithi046.stdout:Preconfiguring packages ... 2023-12-16T21:40:55.882 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package flex. 2023-12-16T21:40:55.989 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173624 files and directories currently installed.) 2023-12-16T21:40:55.994 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-12-16T21:40:56.009 INFO:teuthology.orchestra.run.smithi046.stdout:Fetched 54.6 MB in 1s (43.4 MB/s) 2023-12-16T21:40:56.046 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking flex (2.6.4-8build2) ... 2023-12-16T21:40:56.148 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package flex. 2023-12-16T21:40:56.243 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173624 files and directories currently installed.) 2023-12-16T21:40:56.249 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-12-16T21:40:56.297 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking flex (2.6.4-8build2) ... 2023-12-16T21:40:56.571 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-16T21:40:56.753 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-12-16T21:40:56.988 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-16T21:40:57.146 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-16T21:40:57.194 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-12-16T21:40:57.622 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-16T21:40:57.637 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package bison. 2023-12-16T21:40:57.736 INFO:teuthology.orchestra.run.smithi161.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173697 files and directories currently installed.) 2023-12-16T21:40:57.741 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-12-16T21:40:57.781 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-12-16T21:40:58.139 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-12-16T21:40:58.154 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package bison. 2023-12-16T21:40:58.167 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-12-16T21:40:58.197 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-12-16T21:40:58.261 INFO:teuthology.orchestra.run.smithi046.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173697 files and directories currently installed.) 2023-12-16T21:40:58.267 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-12-16T21:40:58.314 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-12-16T21:40:58.526 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package dump. 2023-12-16T21:40:58.553 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-12-16T21:40:58.592 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking dump (0.4b47-3) ... 2023-12-16T21:40:58.715 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-12-16T21:40:58.743 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-12-16T21:40:58.781 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-12-16T21:40:58.871 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package indent. 2023-12-16T21:40:58.887 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:40:58.929 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-16T21:40:59.084 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package dump. 2023-12-16T21:40:59.112 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-12-16T21:40:59.142 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking dump (0.4b47-3) ... 2023-12-16T21:40:59.181 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-12-16T21:40:59.197 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-12-16T21:40:59.231 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-16T21:40:59.447 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package indent. 2023-12-16T21:40:59.475 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:40:59.521 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-16T21:40:59.560 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-12-16T21:40:59.587 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2023-12-16T21:40:59.627 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-16T21:40:59.789 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-12-16T21:40:59.806 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-12-16T21:40:59.848 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-16T21:40:59.888 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-12-16T21:40:59.908 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-12-16T21:40:59.946 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-12-16T21:41:00.185 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-12-16T21:41:00.213 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2023-12-16T21:41:00.241 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libfl2:amd64. 2023-12-16T21:41:00.251 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-16T21:41:00.268 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-12-16T21:41:00.299 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:00.480 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-12-16T21:41:00.495 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-12-16T21:41:00.502 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-12-16T21:41:00.518 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-12-16T21:41:00.529 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-12-16T21:41:00.551 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:00.804 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libjson-perl. 2023-12-16T21:41:00.832 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-12-16T21:41:00.870 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-12-16T21:41:00.873 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl2:amd64. 2023-12-16T21:41:00.902 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-12-16T21:41:00.940 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:01.182 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libllvm14:amd64. 2023-12-16T21:41:01.192 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-12-16T21:41:01.210 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-12-16T21:41:01.221 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-12-16T21:41:01.248 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-16T21:41:01.284 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:01.528 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libjson-perl. 2023-12-16T21:41:01.545 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-12-16T21:41:01.586 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-12-16T21:41:01.848 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libllvm14:amd64. 2023-12-16T21:41:01.864 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-12-16T21:41:01.906 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-16T21:41:02.809 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-12-16T21:41:02.826 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-16T21:41:02.859 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:03.120 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmbim-proxy. 2023-12-16T21:41:03.136 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-16T21:41:03.178 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:03.506 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-12-16T21:41:03.525 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-12-16T21:41:03.534 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-16T21:41:03.554 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-16T21:41:03.572 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:03.592 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:03.828 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmbim-proxy. 2023-12-16T21:41:03.844 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-16T21:41:03.886 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:03.892 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libndp0:amd64. 2023-12-16T21:41:03.920 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-12-16T21:41:03.950 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-16T21:41:04.155 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-12-16T21:41:04.183 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-16T21:41:04.222 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:04.287 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libnm0:amd64. 2023-12-16T21:41:04.315 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-12-16T21:41:04.353 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-16T21:41:04.574 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libndp0:amd64. 2023-12-16T21:41:04.603 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-12-16T21:41:04.641 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-16T21:41:04.664 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libpq5:amd64. 2023-12-16T21:41:04.693 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:04.722 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:04.977 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libnm0:amd64. 2023-12-16T21:41:05.005 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-12-16T21:41:05.043 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-16T21:41:05.058 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-12-16T21:41:05.087 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:05.125 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:05.355 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libpq5:amd64. 2023-12-16T21:41:05.383 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:05.413 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:05.419 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libqmi-proxy. 2023-12-16T21:41:05.448 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:05.497 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:05.738 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-12-16T21:41:05.754 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-16T21:41:05.765 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-12-16T21:41:05.788 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-16T21:41:05.794 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:05.832 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:06.134 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libqmi-proxy. 2023-12-16T21:41:06.151 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:06.201 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:06.300 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-12-16T21:41:06.328 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-12-16T21:41:06.366 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-12-16T21:41:06.470 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-12-16T21:41:06.487 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-16T21:41:06.529 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-16T21:41:06.711 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package modemmanager. 2023-12-16T21:41:06.739 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-16T21:41:06.794 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:07.023 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-12-16T21:41:07.052 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-12-16T21:41:07.090 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-12-16T21:41:07.392 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package network-manager. 2023-12-16T21:41:07.420 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-12-16T21:41:07.442 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package modemmanager. 2023-12-16T21:41:07.467 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-12-16T21:41:07.470 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-16T21:41:07.509 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:08.057 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package ppp. 2023-12-16T21:41:08.086 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-12-16T21:41:08.090 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager. 2023-12-16T21:41:08.118 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-12-16T21:41:08.165 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-12-16T21:41:08.377 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-12-16T21:41:08.687 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package pptp-linux. 2023-12-16T21:41:08.715 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-12-16T21:41:08.753 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-12-16T21:41:08.822 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package ppp. 2023-12-16T21:41:08.850 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-12-16T21:41:09.006 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package network-manager-pptp. 2023-12-16T21:41:09.022 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-12-16T21:41:09.056 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-12-16T21:41:09.118 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-12-16T21:41:09.342 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-client-common. 2023-12-16T21:41:09.370 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-12-16T21:41:09.409 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-client-common (238) ... 2023-12-16T21:41:09.451 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package pptp-linux. 2023-12-16T21:41:09.467 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-12-16T21:41:09.509 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-12-16T21:41:09.713 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-client-14. 2023-12-16T21:41:09.740 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:09.762 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package network-manager-pptp. 2023-12-16T21:41:09.778 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-12-16T21:41:09.779 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:09.820 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-12-16T21:41:10.098 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-common. 2023-12-16T21:41:10.126 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-12-16T21:41:10.164 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-common (238) ... 2023-12-16T21:41:10.359 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-common. 2023-12-16T21:41:10.387 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-12-16T21:41:10.443 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client-14. 2023-12-16T21:41:10.444 INFO:teuthology.orchestra.run.smithi161.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-12-16T21:41:10.471 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:10.473 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-common (238) ... 2023-12-16T21:41:10.509 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:10.891 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-14. 2023-12-16T21:41:10.919 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:10.953 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:11.131 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-common. 2023-12-16T21:41:11.159 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-12-16T21:41:11.217 INFO:teuthology.orchestra.run.smithi046.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-12-16T21:41:11.237 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-common (238) ... 2023-12-16T21:41:11.705 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-14. 2023-12-16T21:41:11.733 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-16T21:41:11.792 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:12.394 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql. 2023-12-16T21:41:12.423 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-12-16T21:41:12.460 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql (14+238) ... 2023-12-16T21:41:12.721 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-client. 2023-12-16T21:41:12.738 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-12-16T21:41:12.779 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-client (14+238) ... 2023-12-16T21:41:13.023 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package postgresql-contrib. 2023-12-16T21:41:13.032 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql. 2023-12-16T21:41:13.040 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-12-16T21:41:13.050 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-12-16T21:41:13.081 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking postgresql-contrib (14+238) ... 2023-12-16T21:41:13.082 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql (14+238) ... 2023-12-16T21:41:13.301 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-client. 2023-12-16T21:41:13.318 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-12-16T21:41:13.359 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-client (14+238) ... 2023-12-16T21:41:13.367 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package tcl8.6. 2023-12-16T21:41:13.395 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-12-16T21:41:13.434 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-16T21:41:13.603 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package postgresql-contrib. 2023-12-16T21:41:13.632 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-12-16T21:41:13.670 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking postgresql-contrib (14+238) ... 2023-12-16T21:41:13.728 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package tcl. 2023-12-16T21:41:13.756 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-12-16T21:41:13.800 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-12-16T21:41:13.989 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl8.6. 2023-12-16T21:41:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-12-16T21:41:14.055 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-16T21:41:14.148 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package wpasupplicant. 2023-12-16T21:41:14.176 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-12-16T21:41:14.214 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-16T21:41:14.333 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package tcl. 2023-12-16T21:41:14.361 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-12-16T21:41:14.404 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-12-16T21:41:14.585 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-12-16T21:41:14.610 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-12-16T21:41:14.653 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-12-16T21:41:14.719 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package wpasupplicant. 2023-12-16T21:41:14.747 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-12-16T21:41:14.827 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-16T21:41:15.013 INFO:teuthology.orchestra.run.smithi161.stdout:Selecting previously unselected package usb-modeswitch. 2023-12-16T21:41:15.042 INFO:teuthology.orchestra.run.smithi161.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-12-16T21:41:15.089 INFO:teuthology.orchestra.run.smithi161.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-16T21:41:15.215 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-12-16T21:41:15.244 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-12-16T21:41:15.281 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-12-16T21:41:15.377 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-client-common (238) ... 2023-12-16T21:41:15.586 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up flex (2.6.4-8build2) ... 2023-12-16T21:41:15.592 INFO:teuthology.orchestra.run.smithi046.stdout:Selecting previously unselected package usb-modeswitch. 2023-12-16T21:41:15.621 INFO:teuthology.orchestra.run.smithi046.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-12-16T21:41:15.659 INFO:teuthology.orchestra.run.smithi046.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-16T21:41:15.720 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-16T21:41:15.847 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:15.964 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-common (238) ... 2023-12-16T21:41:15.972 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-12-16T21:41:16.082 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-16T21:41:16.191 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:16.298 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up flex (2.6.4-8build2) ... 2023-12-16T21:41:16.407 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-16T21:41:16.508 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:16.618 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-12-16T21:41:16.678 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-12-16T21:41:16.752 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-16T21:41:16.764 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-16T21:41:16.878 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:17.108 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-16T21:41:17.108 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-16T21:41:17.433 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-12-16T21:41:17.525 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-16T21:41:17.583 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-12-16T21:41:17.701 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-16T21:41:17.827 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-12-16T21:41:17.894 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-16T21:41:17.894 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-16T21:41:17.953 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-12-16T21:41:18.088 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:18.230 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-16T21:41:18.353 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-12-16T21:41:18.381 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-12-16T21:41:18.479 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-16T21:41:18.605 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-12-16T21:41:18.731 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-12-16T21:41:18.840 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:18.941 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-16T21:41:18.961 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-16T21:41:19.050 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-12-16T21:41:19.096 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up dump (0.4b47-3) ... 2023-12-16T21:41:19.189 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-12-16T21:41:19.255 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:19.373 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-16T21:41:19.474 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up tcl (8.6.11+1build2) ... 2023-12-16T21:41:19.600 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libjson-perl (4.04000-1) ... 2023-12-16T21:41:19.705 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-16T21:41:19.726 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-12-16T21:41:19.816 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-12-16T21:41:19.831 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up dump (0.4b47-3) ... 2023-12-16T21:41:19.886 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-16T21:41:19.908 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-12-16T21:41:19.966 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:20.014 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-16T21:41:20.075 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-16T21:41:20.182 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-16T21:41:20.201 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up tcl (8.6.11+1build2) ... 2023-12-16T21:41:20.291 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:20.336 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libjson-perl (4.04000-1) ... 2023-12-16T21:41:20.417 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-12-16T21:41:20.462 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-12-16T21:41:20.551 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-12-16T21:41:20.585 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-client (14+238) ... 2023-12-16T21:41:20.621 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-16T21:41:20.711 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-12-16T21:41:20.747 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-16T21:41:20.915 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-16T21:41:21.050 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:21.150 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-12-16T21:41:21.167 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:21.167 INFO:teuthology.orchestra.run.smithi161.stdout:The following network interfaces were found in /etc/network/interfaces 2023-12-16T21:41:21.167 INFO:teuthology.orchestra.run.smithi161.stdout:which means they are currently configured by ifupdown: 2023-12-16T21:41:21.168 INFO:teuthology.orchestra.run.smithi161.stdout:- enp3s0f1 2023-12-16T21:41:21.168 INFO:teuthology.orchestra.run.smithi161.stdout:If you want to manage those interfaces with NetworkManager instead 2023-12-16T21:41:21.168 INFO:teuthology.orchestra.run.smithi161.stdout:remove their configuration from /etc/network/interfaces. 2023-12-16T21:41:21.168 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:21.277 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-12-16T21:41:21.582 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-12-16T21:41:21.727 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-client (14+238) ... 2023-12-16T21:41:21.861 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-12-16T21:41:21.890 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-12-16T21:41:22.338 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:22.338 INFO:teuthology.orchestra.run.smithi046.stdout:The following network interfaces were found in /etc/network/interfaces 2023-12-16T21:41:22.338 INFO:teuthology.orchestra.run.smithi046.stdout:which means they are currently configured by ifupdown: 2023-12-16T21:41:22.338 INFO:teuthology.orchestra.run.smithi046.stdout:- ens1f0 2023-12-16T21:41:22.338 INFO:teuthology.orchestra.run.smithi046.stdout:If you want to manage those interfaces with NetworkManager instead 2023-12-16T21:41:22.338 INFO:teuthology.orchestra.run.smithi046.stdout:remove their configuration from /etc/network/interfaces. 2023-12-16T21:41:22.338 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:22.380 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:22.461 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-12-16T21:41:22.515 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:22.616 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-12-16T21:41:22.733 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-common (238) ... 2023-12-16T21:41:22.796 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-12-16T21:41:23.152 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-12-16T21:41:23.618 INFO:teuthology.orchestra.run.smithi161.stdout:Adding user postgres to group ssl-cert 2023-12-16T21:41:23.664 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-16T21:41:23.807 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-16T21:41:23.835 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:23.835 INFO:teuthology.orchestra.run.smithi161.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-12-16T21:41:23.908 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-12-16T21:41:24.037 INFO:teuthology.orchestra.run.smithi161.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-12-16T21:41:24.037 INFO:teuthology.orchestra.run.smithi161.stdout:Removing obsolete dictionary files: 2023-12-16T21:41:24.042 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-common (238) ... 2023-12-16T21:41:24.571 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-12-16T21:41:24.914 INFO:teuthology.orchestra.run.smithi046.stdout:Adding user postgres to group ssl-cert 2023-12-16T21:41:24.943 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:25.127 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:25.127 INFO:teuthology.orchestra.run.smithi046.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-12-16T21:41:25.262 INFO:teuthology.orchestra.run.smithi161.stdout:Creating new PostgreSQL cluster 14/main ... 2023-12-16T21:41:25.283 INFO:teuthology.orchestra.run.smithi161.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-12-16T21:41:25.324 INFO:teuthology.orchestra.run.smithi046.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-12-16T21:41:25.324 INFO:teuthology.orchestra.run.smithi046.stdout:Removing obsolete dictionary files: 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout:The files belonging to this database system will be owned by user "postgres". 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout:This user must also own the server process. 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout:The database cluster will be initialized with locale "en_US". 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout:The default text search configuration will be set to "english". 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout:Data page checksums are disabled. 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:25.326 INFO:teuthology.orchestra.run.smithi161.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-12-16T21:41:25.327 INFO:teuthology.orchestra.run.smithi161.stdout:creating subdirectories ... ok 2023-12-16T21:41:25.327 INFO:teuthology.orchestra.run.smithi161.stdout:selecting dynamic shared memory implementation ... posix 2023-12-16T21:41:25.339 INFO:teuthology.orchestra.run.smithi161.stdout:selecting default max_connections ... 100 2023-12-16T21:41:25.361 INFO:teuthology.orchestra.run.smithi161.stdout:selecting default shared_buffers ... 128MB 2023-12-16T21:41:25.370 INFO:teuthology.orchestra.run.smithi161.stdout:selecting default time zone ... Etc/UTC 2023-12-16T21:41:25.371 INFO:teuthology.orchestra.run.smithi161.stdout:creating configuration files ... ok 2023-12-16T21:41:25.486 INFO:teuthology.orchestra.run.smithi161.stdout:running bootstrap script ... ok 2023-12-16T21:41:25.894 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-12-16T21:41:26.118 INFO:teuthology.orchestra.run.smithi161.stdout:performing post-bootstrap initialization ... ok 2023-12-16T21:41:26.277 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-16T21:41:26.614 INFO:teuthology.orchestra.run.smithi046.stdout:Creating new PostgreSQL cluster 14/main ... 2023-12-16T21:41:26.630 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-12-16T21:41:26.678 INFO:teuthology.orchestra.run.smithi046.stdout:The files belonging to this database system will be owned by user "postgres". 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:This user must also own the server process. 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:The database cluster will be initialized with locale "en_US". 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:The default text search configuration will be set to "english". 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:Data page checksums are disabled. 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:creating subdirectories ... ok 2023-12-16T21:41:26.679 INFO:teuthology.orchestra.run.smithi046.stdout:selecting dynamic shared memory implementation ... posix 2023-12-16T21:41:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default max_connections ... 100 2023-12-16T21:41:26.711 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default shared_buffers ... 128MB 2023-12-16T21:41:26.721 INFO:teuthology.orchestra.run.smithi046.stdout:selecting default time zone ... Etc/UTC 2023-12-16T21:41:26.721 INFO:teuthology.orchestra.run.smithi046.stdout:creating configuration files ... ok 2023-12-16T21:41:26.821 INFO:teuthology.orchestra.run.smithi161.stdout:syncing data to disk ... ok 2023-12-16T21:41:26.835 INFO:teuthology.orchestra.run.smithi046.stdout:running bootstrap script ... ok 2023-12-16T21:41:27.194 INFO:teuthology.orchestra.run.smithi161.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-12-16T21:41:27.493 INFO:teuthology.orchestra.run.smithi046.stdout:performing post-bootstrap initialization ... ok 2023-12-16T21:41:28.146 INFO:teuthology.orchestra.run.smithi046.stdout:syncing data to disk ... ok 2023-12-16T21:41:28.539 INFO:teuthology.orchestra.run.smithi046.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-12-16T21:41:29.957 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:30.083 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql-contrib (14+238) ... 2023-12-16T21:41:30.209 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up postgresql (14+238) ... 2023-12-16T21:41:30.335 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:30.461 INFO:teuthology.orchestra.run.smithi161.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:30.725 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-12-16T21:41:30.725 INFO:teuthology.orchestra.run.smithi161.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-12-16T21:41:31.255 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-12-16T21:41:31.325 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:31.467 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql-contrib (14+238) ... 2023-12-16T21:41:31.574 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-16T21:41:31.602 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up postgresql (14+238) ... 2023-12-16T21:41:31.725 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-16T21:41:31.728 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-16T21:41:31.854 INFO:teuthology.orchestra.run.smithi046.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-16T21:41:32.116 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-12-16T21:41:32.116 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-12-16T21:41:32.698 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-12-16T21:41:33.003 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-12-16T21:41:33.025 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-16T21:41:33.103 INFO:teuthology.orchestra.run.smithi161.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-16T21:41:33.185 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-16T21:41:33.604 INFO:teuthology.orchestra.run.smithi161.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-16T21:41:33.610 DEBUG:teuthology.parallel:result is None 2023-12-16T21:41:34.557 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-12-16T21:41:34.654 INFO:teuthology.orchestra.run.smithi046.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-16T21:41:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-16T21:41:35.179 DEBUG:teuthology.parallel:result is None 2023-12-16T21:41:35.179 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7586e95519b7be76f3b6e3a4b333888baf46bc9 2023-12-16T21:41:35.418 DEBUG:teuthology.orchestra.run.smithi046:> dpkg-query -W -f '${Version}' ceph 2023-12-16T21:41:35.435 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-202-gc7586e95-1jammy 2023-12-16T21:41:35.435 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-202-gc7586e95-1jammy 2023-12-16T21:41:35.435 INFO:teuthology.task.install:The correct ceph version 19.0.0-202-gc7586e95-1jammy is installed. 2023-12-16T21:41:35.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7586e95519b7be76f3b6e3a4b333888baf46bc9 2023-12-16T21:41:35.583 DEBUG:teuthology.orchestra.run.smithi161:> dpkg-query -W -f '${Version}' ceph 2023-12-16T21:41:35.603 INFO:teuthology.orchestra.run.smithi161.stdout:19.0.0-202-gc7586e95-1jammy 2023-12-16T21:41:35.603 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-202-gc7586e95-1jammy 2023-12-16T21:41:35.603 INFO:teuthology.task.install:The correct ceph version 19.0.0-202-gc7586e95-1jammy is installed. 2023-12-16T21:41:35.605 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-16T21:41:35.605 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:35.605 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-16T21:41:35.620 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:35.620 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-16T21:41:35.657 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-16T21:41:35.657 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:35.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2023-12-16T21:41:35.673 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-16T21:41:35.732 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:35.732 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/daemon-helper 2023-12-16T21:41:35.745 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-16T21:41:35.800 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-16T21:41:35.800 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:35.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-16T21:41:35.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-16T21:41:35.868 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:35.869 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-16T21:41:35.881 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-16T21:41:35.935 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-16T21:41:35.935 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:35.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2023-12-16T21:41:35.949 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-16T21:41:36.005 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:36.005 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/usr/bin/stdin-killer 2023-12-16T21:41:36.019 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-16T21:41:36.072 INFO:teuthology.run_tasks:Running task ceph... 2023-12-16T21:41:36.140 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-16T21:41:36.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /var/log/ceph 2023-12-16T21:41:36.142 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 777 /var/log/ceph 2023-12-16T21:41:36.155 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-16T21:41:36.155 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-16T21:41:36.198 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-16T21:41:36.210 INFO:tasks.ceph:Creating extra log directories... 2023-12-16T21:41:36.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-16T21:41:36.254 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-16T21:41:36.275 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-16T21:41:36.276 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-16T21:41:36.276 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-16T21:41:36.276 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:36.276 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-16T21:41:36.313 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-16T21:41:36.390 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:36.391 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-16T21:41:36.396 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-16T21:41:36.476 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-16T21:41:36.477 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-16T21:41:36.477 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866', 'branch': 'wip-yuri8-testing-2023-12-15-0911', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7494866', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri8-testing-2023-12-15-0911'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri8-testing-2023-12-15-0911', 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 2960, 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri8-testing-2023-12-15-0911', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi046.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwIPeHm8uOK67B7OzSC8aqMeMUOaYZkPomxzmiewZk1jLW5r1rcwjDi/6KEnxjGGz0NDRq/V4VbNnS8qtp6GvU=', 'smithi161.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGXadOoxwcYUpBOvDOHwRreOuQo+PCOSwcoQjSsc62RmCVySnCVdR6sXLE9o3/vznW6S63J92o/drQ/FMSVczVk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'c7586e95519b7be76f3b6e3a4b333888baf46bc9', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73', 'timestamp': '2023-12-16_15:45:14', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078'} 2023-12-16T21:41:36.477 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-16T21:41:36.480 DEBUG:teuthology.orchestra.run.smithi161:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-16T21:41:36.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m0777 -- /var/run/ceph 2023-12-16T21:41:36.526 DEBUG:teuthology.orchestra.run.smithi161:> sudo install -d -m0777 -- /var/run/ceph 2023-12-16T21:41:36.576 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:36.577 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2023-12-16T21:41:36.585 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-16T21:41:36.585 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2023-12-16T21:41:36.633 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-16T21:41:36.633 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 886 Links: 1 2023-12-16T21:41:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-16 21:35:31.657220246 +0000 2023-12-16T21:41:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-16 21:35:31.285223548 +0000 2023-12-16T21:41:36.635 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-16 21:35:31.285223548 +0000 2023-12-16T21:41:36.636 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-12-16T21:41:36.636 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-16T21:41:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-16T21:41:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-16T21:41:36.688 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000308532 s, 1.7 MB/s 2023-12-16T21:41:36.689 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-16T21:41:36.739 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 905 Links: 1 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-16 21:35:32.261214882 +0000 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-16 21:35:31.877218293 +0000 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-16 21:35:31.877218293 +0000 2023-12-16T21:41:36.785 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-12-16T21:41:36.786 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-16T21:41:36.840 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-16T21:41:36.840 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-16T21:41:36.840 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000238636 s, 2.1 MB/s 2023-12-16T21:41:36.841 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-16T21:41:36.890 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2023-12-16T21:41:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-16T21:41:36.937 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:36.937 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 923 Links: 1 2023-12-16T21:41:36.937 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:36.938 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-16 21:35:32.857209592 +0000 2023-12-16T21:41:36.938 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-16 21:35:32.505212717 +0000 2023-12-16T21:41:36.938 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-16 21:35:32.505212717 +0000 2023-12-16T21:41:36.938 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-12-16T21:41:36.938 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-16T21:41:36.992 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-16T21:41:36.992 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-16T21:41:36.992 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000302908 s, 1.7 MB/s 2023-12-16T21:41:36.993 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-16T21:41:37.042 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2023-12-16T21:41:37.089 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-16T21:41:37.089 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:37.089 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 935 Links: 1 2023-12-16T21:41:37.089 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:37.089 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2023-12-16 21:35:33.413204656 +0000 2023-12-16T21:41:37.089 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2023-12-16 21:35:33.073207674 +0000 2023-12-16T21:41:37.090 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2023-12-16 21:35:33.073207674 +0000 2023-12-16T21:41:37.090 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: - 2023-12-16T21:41:37.090 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-16T21:41:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2023-12-16T21:41:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2023-12-16T21:41:37.144 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000308405 s, 1.7 MB/s 2023-12-16T21:41:37.145 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-16T21:41:37.194 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:37.194 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:37.194 DEBUG:teuthology.orchestra.run.smithi161:> dd if=/scratch_devs of=/dev/stdout 2023-12-16T21:41:37.200 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-16T21:41:37.200 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_1 2023-12-16T21:41:37.249 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-16T21:41:37.249 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:37.249 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 875 Links: 1 2023-12-16T21:41:37.249 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:37.249 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-12-16 21:35:26.993882538 +0000 2023-12-16T21:41:37.249 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-12-16 21:35:26.653885576 +0000 2023-12-16T21:41:37.250 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-12-16 21:35:26.653885576 +0000 2023-12-16T21:41:37.250 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-12-16T21:41:37.250 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-16T21:41:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-12-16T21:41:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-12-16T21:41:37.303 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000445937 s, 1.1 MB/s 2023-12-16T21:41:37.304 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-16T21:41:37.354 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_2 2023-12-16T21:41:37.400 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-16T21:41:37.400 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:37.400 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 889 Links: 1 2023-12-16T21:41:37.401 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:37.401 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-12-16 21:35:27.645876716 +0000 2023-12-16T21:41:37.401 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-12-16 21:35:27.269880073 +0000 2023-12-16T21:41:37.401 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-12-16 21:35:27.269880073 +0000 2023-12-16T21:41:37.401 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-12-16T21:41:37.401 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-16T21:41:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-12-16T21:41:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-12-16T21:41:37.454 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000368927 s, 1.4 MB/s 2023-12-16T21:41:37.456 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-16T21:41:37.506 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_3 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 906 Links: 1 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-12-16 21:35:28.217871606 +0000 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-12-16 21:35:27.869874715 +0000 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-12-16 21:35:27.869874715 +0000 2023-12-16T21:41:37.553 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-12-16T21:41:37.553 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-16T21:41:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-12-16T21:41:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-12-16T21:41:37.607 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000345903 s, 1.5 MB/s 2023-12-16T21:41:37.609 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-16T21:41:37.658 DEBUG:teuthology.orchestra.run.smithi161:> stat /dev/vg_nvme/lv_4 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout:Device: 5h/5d Inode: 921 Links: 1 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout:Access: 2023-12-16 21:35:28.829866140 +0000 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout:Modify: 2023-12-16 21:35:28.465869391 +0000 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout:Change: 2023-12-16 21:35:28.465869391 +0000 2023-12-16T21:41:37.705 INFO:teuthology.orchestra.run.smithi161.stdout: Birth: - 2023-12-16T21:41:37.705 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-16T21:41:37.759 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records in 2023-12-16T21:41:37.759 INFO:teuthology.orchestra.run.smithi161.stderr:1+0 records out 2023-12-16T21:41:37.760 INFO:teuthology.orchestra.run.smithi161.stderr:512 bytes copied, 0.000327242 s, 1.6 MB/s 2023-12-16T21:41:37.760 DEBUG:teuthology.orchestra.run.smithi161:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-16T21:41:37.809 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:37.810 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-16T21:41:37.810 INFO:tasks.ceph:Generating config... 2023-12-16T21:41:37.811 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-16T21:41:37.811 INFO:tasks.ceph:[client] debug client = 20 2023-12-16T21:41:37.811 INFO:tasks.ceph:[client] debug ms = 1 2023-12-16T21:41:37.811 INFO:tasks.ceph:[client] pid file = 2023-12-16T21:41:37.811 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-16T21:41:37.811 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-16T21:41:37.811 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-16T21:41:37.811 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-16T21:41:37.812 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-16T21:41:37.812 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-16T21:41:37.812 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-16T21:41:37.812 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-16T21:41:37.813 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-16T21:41:37.814 INFO:tasks.ceph:Setting up mon.a... 2023-12-16T21:41:37.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-16T21:41:37.848 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.keyring 2023-12-16T21:41:37.862 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-16T21:41:37.941 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-16T21:41:37.997 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.46'), ('mon.b', '172.21.15.161'), ('mon.c', '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]')] 2023-12-16T21:41:37.997 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.46,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-16T21:41:37.998 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:37.998 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-16T21:41:38.045 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.46 --add b 172.21.15.161 --addv c '[v2:172.21.15.161:3301,v1:172.21.15.161:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: generated fsid 52c183c6-9df9-436e-b409-f347731c6c1e 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = pacific 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 0 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:fsid 52c183c6-9df9-436e-b409-f347731c6c1e 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:last_changed 2023-12-16T21:41:38.131675+0000 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:created 2023-12-16T21:41:38.131675+0000 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:min_mon_release 16 (pacific) 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:election_strategy: 1 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2023-12-16T21:41:38.133 INFO:teuthology.orchestra.run.smithi046.stdout:1: [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] mon.b 2023-12-16T21:41:38.134 INFO:teuthology.orchestra.run.smithi046.stdout:2: [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] mon.c 2023-12-16T21:41:38.134 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-16T21:41:38.135 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-16T21:41:38.139 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 52c183c6-9df9-436e-b409-f347731c6c1e... 2023-12-16T21:41:38.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-16T21:41:38.182 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-16T21:41:38.208 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2023-12-16T21:41:38.208 INFO:teuthology.orchestra.run.smithi046.stdout: chdir = "" 2023-12-16T21:41:38.208 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: auth supported = cephx 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: filestore xattr use omap = true 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: mon clock drift allowed = 1.000 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: osd crush chooseleaf type = 0 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: auth debug = true 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on old message = true 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: ms die on bug = true 2023-12-16T21:41:38.209 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon pg warn max object skew = 0 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: # disable pg_autoscaler by default for new pools 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default size = 2 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow primary affinity = true 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd allow pg remap = true 2023-12-16T21:41:38.210 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on legacy crush tunables = false 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on crush straw calc version zero = false 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on no sortbitwise = false 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on osd down out interval zero = false 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon warn on too few osds = false 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon_allow_pool_size_one = true 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: osd default data pool replay window = 5 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.211 INFO:teuthology.orchestra.run.smithi046.stdout: mon allow pool delete = true 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: mon cluster log file level = debug 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: debug asserts on shutdown = true 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: mon health detail to clog = false 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: mon host = "172.21.15.46,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = 52c183c6-9df9-436e-b409-f347731c6c1e 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout:[osd] 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: osd journal size = 100 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub load threshold = 5.0 2023-12-16T21:41:38.212 INFO:teuthology.orchestra.run.smithi046.stdout: osd scrub max interval = 600 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd mclock profile = high_recovery_ops 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd recover clone overlap = true 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd recovery max chunk = 1048576 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug shutdown = true 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug op order = true 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug verify stray on activate = true 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug trim objects = true 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd open classes on start = true 2023-12-16T21:41:38.213 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug pg log writeout = true 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: osd deep scrub update digest min age = 30 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: osd map max advance = 10 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: journal zero on create = true 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: filestore ondisk finisher threads = 3 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: filestore apply finisher threads = 3 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: bdev debug aio = true 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: osd debug misdirected ops = true 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: bdev async discard = True 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: bdev enable discard = True 2023-12-16T21:41:38.214 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore allocator = bitmap 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore block size = 96636764160 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: bluestore fsck on mount = True 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluefs = 1/20 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: debug bluestore = 1/20 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: debug osd = 20 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: debug rocksdb = 4/10 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd backfillfull_ratio = 0.85 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd full ratio = 0.9 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd nearfull ratio = 0.8 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: osd failsafe full ratio = 0.95 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: osd objectstore = bluestore 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-12-16T21:41:38.215 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr] 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug mgr = 20 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: mgr/telemetry/nag = false 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout:[mon] 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug mon = 20 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug paxos = 20 2023-12-16T21:41:38.216 INFO:teuthology.orchestra.run.smithi046.stdout: debug auth = 20 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon data avail warn = 5 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon mgr mkfs grace = 240 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min pgs per osd = 4 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd reporter subtree level = osd 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon osd prime pg temp = true 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon reweight min bytes per osd = 10 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: auth mon ticket ttl = 660 # 11m 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: auth service ticket ttl = 240 # 4m 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: # don't complain about insecure global_id in the test suite 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-16T21:41:38.217 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: # 1m isn't quite enough 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon_down_mkfs_grace = 2m 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon_warn_on_filestore_osds = false 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: mon op complaint time = 120 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout:[client] 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: rgw cache enabled = true 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable ops log = true 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: rgw enable usage log = true 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: client mount timeout = 600 2023-12-16T21:41:38.218 INFO:teuthology.orchestra.run.smithi046.stdout: debug client = 20 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: pid file = "" 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.a] 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.b] 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout:[mon.c] 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout:[mds] 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds = 20 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: debug mds balancer = 20 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: debug ms = 1 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug frag = True 2023-12-16T21:41:38.219 INFO:teuthology.orchestra.run.smithi046.stdout: mds debug scatterstat = True 2023-12-16T21:41:38.220 INFO:teuthology.orchestra.run.smithi046.stdout: mds op complaint time = 180 2023-12-16T21:41:38.220 INFO:teuthology.orchestra.run.smithi046.stdout: mds verify scatter = True 2023-12-16T21:41:38.220 INFO:teuthology.orchestra.run.smithi046.stdout: osd op complaint time = 180 2023-12-16T21:41:38.220 INFO:teuthology.orchestra.run.smithi046.stdout: rados mon op timeout = 900 2023-12-16T21:41:38.220 INFO:teuthology.orchestra.run.smithi046.stdout: rados osd op timeout = 900 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout:[global] 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: chdir = "" 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: auth supported = cephx 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: filestore xattr use omap = true 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: mon clock drift allowed = 1.000 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: osd crush chooseleaf type = 0 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: auth debug = true 2023-12-16T21:41:38.221 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on old message = true 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: ms die on bug = true 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: mon pg warn max object skew = 0 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: # disable pg_autoscaler by default for new pools 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default size = 2 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow primary affinity = true 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd allow pg remap = true 2023-12-16T21:41:38.222 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on legacy crush tunables = false 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on crush straw calc version zero = false 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on no sortbitwise = false 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on osd down out interval zero = false 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: mon warn on too few osds = false 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: mon_allow_pool_size_one = true 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-16T21:41:38.223 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: osd default data pool replay window = 5 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: mon allow pool delete = true 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: mon cluster log file level = debug 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: debug asserts on shutdown = true 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: mon health detail to clog = false 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: mon host = "172.21.15.46,172.21.15.161,[v2:172.21.15.161:3301,v1:172.21.15.161:6790]" 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: fsid = 52c183c6-9df9-436e-b409-f347731c6c1e 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout:[osd] 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: osd journal size = 100 2023-12-16T21:41:38.224 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub load threshold = 5.0 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd scrub max interval = 600 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd mclock profile = high_recovery_ops 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd recover clone overlap = true 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd recovery max chunk = 1048576 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug shutdown = true 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug op order = true 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug verify stray on activate = true 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug trim objects = true 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.225 INFO:teuthology.orchestra.run.smithi161.stdout: osd open classes on start = true 2023-12-16T21:41:38.226 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug pg log writeout = true 2023-12-16T21:41:38.226 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.226 INFO:teuthology.orchestra.run.smithi161.stdout: osd deep scrub update digest min age = 30 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: osd map max advance = 10 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: journal zero on create = true 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: filestore ondisk finisher threads = 3 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: filestore apply finisher threads = 3 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: bdev debug aio = true 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: osd debug misdirected ops = true 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: bdev async discard = True 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: bdev enable discard = True 2023-12-16T21:41:38.227 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore allocator = bitmap 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore block size = 96636764160 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: bluestore fsck on mount = True 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluefs = 1/20 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: debug bluestore = 1/20 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: debug osd = 20 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: debug rocksdb = 4/10 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd backfillfull_ratio = 0.85 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd full ratio = 0.9 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd nearfull ratio = 0.8 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: osd failsafe full ratio = 0.95 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: osd objectstore = bluestore 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2023-12-16T21:41:38.228 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr] 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout: debug mgr = 20 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2023-12-16T21:41:38.229 INFO:teuthology.orchestra.run.smithi161.stdout: mgr/telemetry/nag = false 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout:[mon] 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: debug mon = 20 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: debug paxos = 20 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: debug auth = 20 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: mon data avail warn = 5 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: mon mgr mkfs grace = 240 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min pgs per osd = 4 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd reporter subtree level = osd 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: mon osd prime pg temp = true 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: mon reweight min bytes per osd = 10 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.230 INFO:teuthology.orchestra.run.smithi161.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: auth mon ticket ttl = 660 # 11m 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: auth service ticket ttl = 240 # 4m 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: # don't complain about insecure global_id in the test suite 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: # 1m isn't quite enough 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: mon_down_mkfs_grace = 2m 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.231 INFO:teuthology.orchestra.run.smithi161.stdout: mon_warn_on_filestore_osds = false 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: mon op complaint time = 120 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout:[client] 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: rgw cache enabled = true 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable ops log = true 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: rgw enable usage log = true 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: client mount timeout = 600 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: debug client = 20 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: pid file = "" 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2023-12-16T21:41:38.232 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.a] 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.b] 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout:[mon.c] 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout:[mds] 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds = 20 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: debug mds balancer = 20 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: debug ms = 1 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug frag = True 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: mds debug scatterstat = True 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: mds op complaint time = 180 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: mds verify scatter = True 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: osd op complaint time = 180 2023-12-16T21:41:38.233 INFO:teuthology.orchestra.run.smithi161.stdout: rados mon op timeout = 900 2023-12-16T21:41:38.234 INFO:teuthology.orchestra.run.smithi161.stdout: rados osd op timeout = 900 2023-12-16T21:41:38.235 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-16T21:41:38.236 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-16T21:41:38.325 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-16T21:41:38.325 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:38.325 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-16T21:41:38.330 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:38.331 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-16T21:41:38.376 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:41:38.376 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:38.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-16T21:41:38.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-16T21:41:38.438 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:41:38.438 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:38.484 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:41:38.485 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:38.485 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-16T21:41:38.485 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-16T21:41:38.508 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:41:38.508 DEBUG:teuthology.orchestra.run.smithi161:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:38.557 INFO:tasks.ceph:Setting up mon nodes... 2023-12-16T21:41:38.557 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-16T21:41:38.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-16T21:41:38.603 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-16T21:41:38.606 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-16T21:41:38.648 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-16T21:41:38.662 INFO:tasks.ceph:Setting up mds nodes... 2023-12-16T21:41:38.662 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-16T21:41:38.704 INFO:teuthology.orchestra.run.smithi046.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-16T21:41:38.707 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-16T21:41:38.761 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-16T21:41:38.803 INFO:teuthology.orchestra.run.smithi161.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-16T21:41:38.807 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-16T21:41:38.860 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-16T21:41:38.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-16T21:41:38.896 INFO:teuthology.orchestra.run.smithi046.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-16T21:41:38.907 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-16T21:41:38.907 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi046.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi161.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-16T21:41:38.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-16T21:41:38.960 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:38.960 INFO:tasks.ceph:role: osd.0 2023-12-16T21:41:38.961 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:41:38.961 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:39.020 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:39.044 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-16T21:41:39.046 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-16T21:41:39.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-16T21:41:39.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-16T21:41:39.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:39.162 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:39.163 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-16T21:41:39.217 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:39.217 INFO:tasks.ceph:role: osd.1 2023-12-16T21:41:39.217 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:41:39.217 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-16T21:41:39.273 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:39.273 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:39.274 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:39.297 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-16T21:41:39.299 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-16T21:41:39.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-16T21:41:39.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-16T21:41:39.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:39.410 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:39.410 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-16T21:41:39.465 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:39.465 INFO:tasks.ceph:role: osd.2 2023-12-16T21:41:39.465 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:41:39.465 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-16T21:41:39.521 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:39.522 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:39.546 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-16T21:41:39.548 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-16T21:41:39.548 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-16T21:41:39.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-16T21:41:39.664 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:39.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:39.666 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-16T21:41:39.720 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:39.720 INFO:tasks.ceph:role: osd.3 2023-12-16T21:41:39.720 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:41:39.721 DEBUG:teuthology.orchestra.run.smithi046:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-16T21:41:39.777 INFO:teuthology.orchestra.run.smithi046.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:39.777 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:39.777 INFO:teuthology.orchestra.run.smithi046.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:39.777 INFO:teuthology.orchestra.run.smithi046.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:39.778 INFO:teuthology.orchestra.run.smithi046.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:39.778 INFO:teuthology.orchestra.run.smithi046.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:39.778 INFO:teuthology.orchestra.run.smithi046.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:39.778 INFO:teuthology.orchestra.run.smithi046.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:39.778 INFO:teuthology.orchestra.run.smithi046.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:39.778 INFO:teuthology.orchestra.run.smithi046.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:39.801 INFO:teuthology.orchestra.run.smithi046.stdout:Discarding blocks...Done. 2023-12-16T21:41:39.804 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi046.front.sepia.ceph.com -o noatime 2023-12-16T21:41:39.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-16T21:41:39.870 DEBUG:teuthology.orchestra.run.smithi046:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-16T21:41:39.923 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:39.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:39.924 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:40.048+0000 7efefbfdc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-16T21:41:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:40.048+0000 7efefbfdc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-16T21:41:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:40.048+0000 7efefbfdc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-16T21:41:40.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:40.048+0000 7efefbfdc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-16T21:41:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:40.048+0000 7efefbfdc640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-16T21:41:40.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:40.048+0000 7efefbfdc640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-16T21:41:43.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-16T21:41:43.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:43.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:43.180+0000 7fab5afb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-16T21:41:43.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:43.180+0000 7fab5afb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-16T21:41:43.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:43.180+0000 7fab5afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-16T21:41:43.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:43.180+0000 7fab5afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-16T21:41:43.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:43.180+0000 7fab5afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-16T21:41:43.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:43.180+0000 7fab5afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-16T21:41:46.185 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-16T21:41:46.246 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:46.324+0000 7f8820ba8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-16T21:41:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:46.324+0000 7f8820ba8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-16T21:41:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:46.324+0000 7f8820ba8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-16T21:41:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:46.324+0000 7f8820ba8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-16T21:41:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:46.324+0000 7f8820ba8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-16T21:41:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:46.324+0000 7f8820ba8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-16T21:41:49.330 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-16T21:41:49.389 DEBUG:teuthology.orchestra.run.smithi046:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:49.468+0000 7f5448aa6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-16T21:41:49.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:49.468+0000 7f5448aa6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-16T21:41:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:49.468+0000 7f5448aa6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-16T21:41:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:49.468+0000 7f5448aa6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-16T21:41:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:49.468+0000 7f5448aa6640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-16T21:41:49.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:41:49.468+0000 7f5448aa6640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-16T21:41:52.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-16T21:41:52.522 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-16T21:41:52.536 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:52.536 INFO:tasks.ceph:role: osd.4 2023-12-16T21:41:52.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:41:52.536 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:52.603 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:53.075 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-12-16T21:41:53.077 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-12-16T21:41:53.077 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-16T21:41:53.243 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-16T21:41:53.254 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:53.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:53.255 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-16T21:41:53.308 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:53.308 INFO:tasks.ceph:role: osd.5 2023-12-16T21:41:53.308 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:41:53.308 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-16T21:41:53.371 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:53.371 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:53.372 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:53.372 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:53.372 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:53.372 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:53.372 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:53.822 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-12-16T21:41:53.824 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-12-16T21:41:53.825 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-16T21:41:53.906 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-16T21:41:53.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:53.958 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:53.958 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-16T21:41:54.012 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:54.012 INFO:tasks.ceph:role: osd.6 2023-12-16T21:41:54.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:41:54.012 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-16T21:41:54.075 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:54.076 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:54.558 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-12-16T21:41:54.560 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-12-16T21:41:54.561 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-16T21:41:54.693 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-16T21:41:54.703 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:54.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:54.704 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-16T21:41:54.759 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-16T21:41:54.759 INFO:tasks.ceph:role: osd.7 2023-12-16T21:41:54.759 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:41:54.759 DEBUG:teuthology.orchestra.run.smithi161:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-16T21:41:54.819 INFO:teuthology.orchestra.run.smithi161.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-16T21:41:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-16T21:41:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-16T21:41:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-16T21:41:54.819 INFO:teuthology.orchestra.run.smithi161.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-16T21:41:54.819 INFO:teuthology.orchestra.run.smithi161.stdout: = sunit=0 swidth=0 blks 2023-12-16T21:41:54.820 INFO:teuthology.orchestra.run.smithi161.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-16T21:41:54.820 INFO:teuthology.orchestra.run.smithi161.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-16T21:41:54.820 INFO:teuthology.orchestra.run.smithi161.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-16T21:41:54.820 INFO:teuthology.orchestra.run.smithi161.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-16T21:41:55.320 INFO:teuthology.orchestra.run.smithi161.stdout:Discarding blocks...Done. 2023-12-16T21:41:55.322 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi161.front.sepia.ceph.com -o noatime 2023-12-16T21:41:55.322 DEBUG:teuthology.orchestra.run.smithi161:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-16T21:41:55.484 DEBUG:teuthology.orchestra.run.smithi161:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-16T21:41:55.495 INFO:teuthology.orchestra.run.smithi161.stderr:sudo: /sbin/restorecon: command not found 2023-12-16T21:41:55.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:41:55.496 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:55.615 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:55.610+0000 7fee62bc4640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-16T21:41:55.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:55.614+0000 7fee62bc4640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-16T21:41:55.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:55.614+0000 7fee62bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-16T21:41:55.617 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:55.614+0000 7fee62bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-16T21:41:55.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:55.614+0000 7fee62bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-16T21:41:55.618 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:55.614+0000 7fee62bc4640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-16T21:41:58.627 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-16T21:41:58.641 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:41:58.723 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:58.718+0000 7f3643b2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-16T21:41:58.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:58.722+0000 7f3643b2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-16T21:41:58.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:58.722+0000 7f3643b2c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-16T21:41:58.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:58.722+0000 7f3643b2c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-16T21:41:58.724 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:58.722+0000 7f3643b2c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-16T21:41:58.725 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:41:58.722+0000 7f3643b2c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-16T21:42:01.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-16T21:42:01.821 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:42:01.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:01.898+0000 7f0a6de56640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-16T21:42:01.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:01.898+0000 7f0a6de56640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-16T21:42:01.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:01.898+0000 7f0a6de56640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-16T21:42:01.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:01.898+0000 7f0a6de56640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-16T21:42:01.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:01.898+0000 7f0a6de56640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-16T21:42:01.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:01.898+0000 7f0a6de56640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-16T21:42:04.899 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-16T21:42:04.957 DEBUG:teuthology.orchestra.run.smithi161:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:42:05.039 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:05.034+0000 7fdd7cc9c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-16T21:42:05.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:05.038+0000 7fdd7cc9c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-16T21:42:05.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:05.038+0000 7fdd7cc9c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-16T21:42:05.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:05.038+0000 7fdd7cc9c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-16T21:42:05.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:05.038+0000 7fdd7cc9c640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-16T21:42:05.040 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:05.038+0000 7fdd7cc9c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-16T21:42:06.479 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:42:06.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:42:08.059 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-16T21:42:08.073 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-16T21:42:08.073 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:08.073 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-16T21:42:08.087 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:08.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-16T21:42:08.142 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:08.142 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-16T21:42:08.196 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:08.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-16T21:42:08.252 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:08.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-16T21:42:08.309 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:08.309 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-16T21:42:08.364 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:08.364 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-16T21:42:08.378 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:08.378 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-16T21:42:08.434 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:08.435 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-16T21:42:08.487 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:08.487 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-16T21:42:08.543 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:08.543 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-16T21:42:08.598 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:08.599 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-16T21:42:08.651 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:08.651 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-16T21:42:08.657 INFO:tasks.ceph:Adding keys to all mons... 2023-12-16T21:42:08.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-16T21:42:08.702 DEBUG:teuthology.orchestra.run.smithi161:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-16T21:42:08.711 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.y] 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSGX5lPwTqIxAAvyqGdtQlV+u1hf32PLcwzA== 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a] 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSGX5lBefnKRAAnNh84aNm6CBfCEA6sfo8bw== 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.0] 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCUGX5lMAwnAxAADVv9fIbJsTxnHtiyQxarSw== 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.1] 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCXGX5llUz8ChAAZJexX0tCjCLl6tVI/yArWQ== 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.2] 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCaGX5lcDSaExAAgAnW1qGMIg5PVqi4AW0Gwg== 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.3] 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCdGX5lhpIiHBAAiNCrUEeKg81BaM7c7PlTIw== 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout:[mgr.x] 2023-12-16T21:42:08.712 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSGX5lR1isJhAAfuAWAQcooIEu0EhCOvyIpQ== 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b] 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSGX5lrxzsLxAANfoHUyBQNab5129OdBJH0Q== 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.4] 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCjGX5lGB25JBAAo+/p53vX+K7QmkeTH3ImcQ== 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.5] 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCmGX5l9tIsKxAA2+qYGxVKxakKzPLUQFX3rg== 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.6] 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCpGX5l+grMNRAAFoWyoAatVVmOueZ/qFVosA== 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout:[osd.7] 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCtGX5l8BphAhAAcENaFhTC3yX5ilka5mhS5g== 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-16T21:42:08.713 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSGX5lKFNZNRAAtR1lv++0meNbUxdnBSKAGg== 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.y] 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCSGX5lPwTqIxAAvyqGdtQlV+u1hf32PLcwzA== 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.a] 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCSGX5lBefnKRAAnNh84aNm6CBfCEA6sfo8bw== 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.0] 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCUGX5lMAwnAxAADVv9fIbJsTxnHtiyQxarSw== 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.1] 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCXGX5llUz8ChAAZJexX0tCjCLl6tVI/yArWQ== 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.2] 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCaGX5lcDSaExAAgAnW1qGMIg5PVqi4AW0Gwg== 2023-12-16T21:42:08.715 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.3] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCdGX5lhpIiHBAAiNCrUEeKg81BaM7c7PlTIw== 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout:[mgr.x] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCSGX5lR1isJhAAfuAWAQcooIEu0EhCOvyIpQ== 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout:[mds.b] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCSGX5lrxzsLxAANfoHUyBQNab5129OdBJH0Q== 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.4] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCjGX5lGB25JBAAo+/p53vX+K7QmkeTH3ImcQ== 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.5] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCmGX5l9tIsKxAA2+qYGxVKxakKzPLUQFX3rg== 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.6] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCpGX5l+grMNRAAFoWyoAatVVmOueZ/qFVosA== 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout:[osd.7] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCtGX5l8BphAhAAcENaFhTC3yX5ilka5mhS5g== 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout:[client.0] 2023-12-16T21:42:08.716 INFO:teuthology.orchestra.run.smithi161.stdout: key = AQCSGX5lKFNZNRAAtR1lv++0meNbUxdnBSKAGg== 2023-12-16T21:42:08.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-16T21:42:08.760 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-16T21:42:08.805 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-16T21:42:08.846 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-16T21:42:08.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:08.894 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:08.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:08.978 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.060 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.063 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.109 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.182 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-16T21:42:09.194 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-16T21:42:09.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-16T21:42:09.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-16T21:42:09.346 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.366 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.450 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.534 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.618 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-16T21:42:09.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-16T21:42:09.673 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-16T21:42:09.755 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-16T21:42:09.755 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-16T21:42:09.768 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-16T21:42:09.868 DEBUG:teuthology.orchestra.run.smithi046:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-16T21:42:09.880 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-16T21:42:09.893 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-16T21:42:09.991 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-16T21:42:10.002 DEBUG:teuthology.orchestra.run.smithi161:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-16T21:42:10.055 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-16T21:42:10.145 DEBUG:teuthology.orchestra.run.smithi161:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-16T21:42:10.155 DEBUG:teuthology.orchestra.run.smithi046:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:42:10.158 DEBUG:teuthology.orchestra.run.smithi161:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-16T21:42:10.207 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-16T21:42:10.207 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-16T21:42:10.207 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-16T21:42:10.210 INFO:tasks.ceph.mon.a:Started 2023-12-16T21:42:10.210 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-16T21:42:10.210 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-16T21:42:10.253 INFO:tasks.ceph.mon.b:Started 2023-12-16T21:42:10.253 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-16T21:42:10.253 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-16T21:42:10.255 INFO:tasks.ceph.mon.c:Started 2023-12-16T21:42:10.255 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-16T21:42:10.255 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-16T21:42:10.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-16T21:42:10.258 INFO:tasks.ceph.mgr.y:Started 2023-12-16T21:42:10.258 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-16T21:42:10.258 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-16T21:42:10.260 INFO:tasks.ceph.mgr.x:Started 2023-12-16T21:42:10.261 INFO:tasks.ceph:Setting crush tunables to default 2023-12-16T21:42:10.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph osd crush tunables default 2023-12-16T21:42:10.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e342ff640 1 Processor -- start 2023-12-16T21:42:10.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e342ff640 1 -- start start 2023-12-16T21:42:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e342ff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e342ff640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0b1ba0 con 0x7f7e2c0ba920 2023-12-16T21:42:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e342ff640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0b4b50 con 0x7f7e2c141b60 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e342ff640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c141f90 con 0x7f7e2c0b5ec0 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e3209b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7e2c0ba920 0x7f7e2c147a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:36032/0 (socket says 172.21.15.46:36032) 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 learned_addr learned my addr 172.21.15.46:0/611806385 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.376+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-16T21:42:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.580+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.580+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.580+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.580+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:10.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.580+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-16T21:42:10.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.984+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:10.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.984+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:10.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.984+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:10.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.984+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:10.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:10.984+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-16T21:42:11.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:11.788+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:11.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:11.788+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:11.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:11.788+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:11.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:11.788+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:11.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:11.788+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-16T21:42:13.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 msgr2=0x7f7e2c141620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-16T21:42:13.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.46:3300/0 conn(0x7f7e2c0b5ec0 0x7f7e2c141620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:13.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 >> v1:172.21.15.161:6789/0 conn(0x7f7e2c141b60 legacy=0x7f7e2c14ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:13.382 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 >> v1:172.21.15.46:6789/0 conn(0x7f7e2c0ba920 legacy=0x7f7e2c147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c141f90 con 0x7f7e100036f0 2023-12-16T21:42:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e10008d80 con 0x7f7e10006190 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-16T21:42:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.376+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-16T21:42:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-16T21:42:13.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-16T21:42:13.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.580+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-16T21:42:13.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:13.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:13.984+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-16T21:42:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-16T21:42:14.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3189a640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-16T21:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-16T21:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-16T21:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-16T21:42:14.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:14.788+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-16T21:42:15.943 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:15.938+0000 7f623559d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-16T21:42:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 msgr2=0x7f7e10008650 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-16T21:42:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e10006190 0x7f7e10008650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e10005b50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e10005b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3289c640 1 -- 172.21.15.46:0/611806385 reap_dead start 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e100100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0b6350 con 0x7f7e2c0ba920 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e10005c40 con 0x7f7e2c141b60 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e23fff640 1 -- 172.21.15.46:0/611806385 --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c141f90 con 0x7f7e100036f0 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3289c640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e100100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990500791 0 0) 0x7f7e2c0b6350 con 0x7f7e2c0ba920 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e18003530 con 0x7f7e2c0ba920 2023-12-16T21:42:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 137361967 0 0) 0x7f7e18003530 con 0x7f7e2c0ba920 2023-12-16T21:42:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 msgr2=0x7f7e100100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 --2- 172.21.15.46:0/611806385 >> v2:172.21.15.161:3300/0 conn(0x7f7e100036f0 0x7f7e100100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 >> v1:172.21.15.161:6789/0 conn(0x7f7e2c141b60 legacy=0x7f7e1000e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10005dd0 con 0x7f7e2c0ba920 2023-12-16T21:42:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f7e24001e10 con 0x7f7e2c0ba920 2023-12-16T21:42:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 >> v1:172.21.15.46:6789/0 conn(0x7f7e2c0ba920 legacy=0x7f7e1000b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e18003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e180047d0 0x7f7e18004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e180051e0 0x7f7e1800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c141f90 con 0x7f7e18003980 2023-12-16T21:42:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1800c130 con 0x7f7e180051e0 2023-12-16T21:42:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1800c2f0 con 0x7f7e180047d0 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e18003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3289c640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e180051e0 0x7f7e1800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3189a640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e180047d0 0x7f7e18004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e180047d0 msgr2=0x7f7e18004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e180047d0 0x7f7e18004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e180051e0 msgr2=0x7f7e1800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e180051e0 0x7f7e1800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.388 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3209b640 1 -- 172.21.15.46:0/611806385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e18003530 con 0x7f7e18003980 2023-12-16T21:42:16.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.376+0000 7f7e3289c640 1 -- 172.21.15.46:0/611806385 reap_dead start 2023-12-16T21:42:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e3209b640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e18003da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009010 tx=0x7f7e2401e290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5b7fcf62db88e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:16.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240af410 con 0x7f7e18003980 2023-12-16T21:42:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e240a9030 con 0x7f7e18003980 2023-12-16T21:42:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e31099640 1 -- 172.21.15.46:0/611806385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e240af5b0 con 0x7f7e18003980 2023-12-16T21:42:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e342ff640 1 -- 172.21.15.46:0/611806385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 msgr2=0x7f7e18003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e342ff640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e18003da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009010 tx=0x7f7e2401e290 comp rx=0 tx=0).stop 2023-12-16T21:42:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e342ff640 1 -- 172.21.15.46:0/611806385 shutdown_connections 2023-12-16T21:42:16.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e342ff640 1 --2- 172.21.15.46:0/611806385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e18003da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e342ff640 1 -- 172.21.15.46:0/611806385 >> 172.21.15.46:0/611806385 conn(0x7f7e2c06ab80 msgr2=0x7f7e2c0b9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e342ff640 1 -- 172.21.15.46:0/611806385 shutdown_connections 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.380+0000 7f7e342ff640 1 -- 172.21.15.46:0/611806385 wait complete. 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 Processor -- start 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 -- start start 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e2c0b9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2c0b3bd0 0x7f7e2c13ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e2c140470 0x7f7e2c1e0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0b3290 con 0x7f7e18003980 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0b2cd0 con 0x7f7e2c140470 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0b3720 con 0x7f7e2c0b3bd0 2023-12-16T21:42:16.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e2c0b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e2c0b9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49496/0 (socket says 172.21.15.46:49496) 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 -- 172.21.15.46:0/1843519467 learned_addr learned my addr 172.21.15.46:0/1843519467 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3189a640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2c0b3bd0 0x7f7e2c13ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3289c640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e2c140470 0x7f7e2c1e0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 -- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2c0b3bd0 msgr2=0x7f7e2c13ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2c0b3bd0 0x7f7e2c13ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 -- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e2c140470 msgr2=0x7f7e2c1e0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e2c140470 0x7f7e2c1e0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 -- 172.21.15.46:0/1843519467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c0b4590 con 0x7f7e18003980 2023-12-16T21:42:16.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e3209b640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e2c0b9060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009010 tx=0x7f7e18005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e22ffd640 1 -- 172.21.15.46:0/1843519467 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24004d80 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e22ffd640 1 -- 172.21.15.46:0/1843519467 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24001b90 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c0b2f20 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e22ffd640 1 -- 172.21.15.46:0/1843519467 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24001f20 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c0b4050 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e22ffd640 1 -- 172.21.15.46:0/1843519467 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7e240b6040 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.384+0000 7f7e22ffd640 1 -- 172.21.15.46:0/1843519467 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7e240c4c00 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.388+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c0b4300 con 0x7f7e18003980 2023-12-16T21:42:16.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.388+0000 7f7e22ffd640 1 -- 172.21.15.46:0/1843519467 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f7e240bf070 con 0x7f7e18003980 2023-12-16T21:42:16.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.452+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7e2c0b4300 con 0x7f7e18003980 2023-12-16T21:42:16.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.456+0000 7f7e22ffd640 1 -- 172.21.15.46:0/1843519467 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7e240b4070 con 0x7f7e18003980 2023-12-16T21:42:16.463 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 msgr2=0x7f7e2c0b9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e2c0b9060 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009010 tx=0x7f7e18005800 comp rx=0 tx=0).stop 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 shutdown_connections 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7e2c140470 0x7f7e2c1e0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7e2c0b3bd0 0x7f7e2c13ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 --2- 172.21.15.46:0/1843519467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e18003980 0x7f7e2c0b9060 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 >> 172.21.15.46:0/1843519467 conn(0x7f7e2c06ab80 msgr2=0x7f7e2c147e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:16.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 shutdown_connections 2023-12-16T21:42:16.466 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:16.460+0000 7f7e342ff640 1 -- 172.21.15.46:0/1843519467 wait complete. 2023-12-16T21:42:16.472 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:16.470+0000 7f623559d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-16T21:42:16.484 INFO:tasks.ceph:check_enable_crimson: False 2023-12-16T21:42:16.484 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-16T21:42:16.484 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:16.484 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-16T21:42:16.496 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:16.496 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-16T21:42:16.556 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:16.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-16T21:42:16.612 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:42:16.612 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-16T21:42:16.635 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:16.628+0000 7ff405085180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-16T21:42:16.647 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:16.647 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-16T21:42:16.658 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:16.654+0000 7f623559d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-16T21:42:16.660 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:16.660 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-16T21:42:16.715 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:16.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-16T21:42:16.770 DEBUG:teuthology.orchestra.run.smithi161:> set -ex 2023-12-16T21:42:16.770 DEBUG:teuthology.orchestra.run.smithi161:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-16T21:42:16.822 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new a6d39371-17a2-4ed3-90aa-eff61ee76362 0 2023-12-16T21:42:16.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.974+0000 7f6f1b86d640 1 Processor -- start 2023-12-16T21:42:16.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.974+0000 7f6f1b86d640 1 -- start start 2023-12-16T21:42:16.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.974+0000 7f6f1b86d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f14143830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6f14143d70 0x7f6f14151080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f14073a90 con 0x7f6f140736b0 2023-12-16T21:42:16.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f140af0e0 con 0x7f6f14143d70 2023-12-16T21:42:16.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f140adb30 con 0x7f6f1406b620 2023-12-16T21:42:16.980 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f18e08640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f14143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f19609640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f6f140736b0 0x7f6f140ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:42342/0 (socket says 172.21.15.161:42342) 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f19609640 1 -- 172.21.15.161:0/3823613748 learned_addr learned my addr 172.21.15.161:0/3823613748 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f0bfff640 1 -- 172.21.15.161:0/3823613748 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343505673 0 0) 0x7f6f14073a90 con 0x7f6f140736b0 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f0bfff640 1 -- 172.21.15.161:0/3823613748 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f00003530 con 0x7f6f140736b0 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f18e08640 1 -- 172.21.15.161:0/3823613748 >> v2:172.21.15.46:3300/0 conn(0x7f6f14143d70 msgr2=0x7f6f14151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f18e08640 1 --2- 172.21.15.161:0/3823613748 >> v2:172.21.15.46:3300/0 conn(0x7f6f14143d70 0x7f6f14151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f18e08640 1 -- 172.21.15.161:0/3823613748 >> v1:172.21.15.161:6789/0 conn(0x7f6f140736b0 legacy=0x7f6f140ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f18e08640 1 -- 172.21.15.161:0/3823613748 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f140ade10 con 0x7f6f1406b620 2023-12-16T21:42:16.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f18e08640 1 --2- 172.21.15.161:0/3823613748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f14143830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f040097c0 tx=0x7f6f0401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b663fa75e22d354 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:16.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f0bfff640 1 -- 172.21.15.161:0/3823613748 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f040ab600 con 0x7f6f1406b620 2023-12-16T21:42:16.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f0bfff640 1 -- 172.21.15.161:0/3823613748 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f040a7050 con 0x7f6f1406b620 2023-12-16T21:42:16.982 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f0bfff640 1 -- 172.21.15.161:0/3823613748 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f040ab980 con 0x7f6f1406b620 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- 172.21.15.161:0/3823613748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 msgr2=0x7f6f14143830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 --2- 172.21.15.161:0/3823613748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f14143830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6f040097c0 tx=0x7f6f0401c3f0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- 172.21.15.161:0/3823613748 shutdown_connections 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 --2- 172.21.15.161:0/3823613748 >> v2:172.21.15.46:3300/0 conn(0x7f6f14143d70 0x7f6f14151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 --2- 172.21.15.161:0/3823613748 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f14143830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- 172.21.15.161:0/3823613748 >> 172.21.15.161:0/3823613748 conn(0x7f6f1406a580 msgr2=0x7f6f1406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- 172.21.15.161:0/3823613748 shutdown_connections 2023-12-16T21:42:16.984 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.978+0000 7f6f1b86d640 1 -- 172.21.15.161:0/3823613748 wait complete. 2023-12-16T21:42:16.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 Processor -- start 2023-12-16T21:42:16.985 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 -- start start 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f141422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f140736b0 0x7f6f14144650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14143d70 0x7f6f14145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14144b90 con 0x7f6f140736b0 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1413b140 con 0x7f6f14143d70 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f141def70 con 0x7f6f1406b620 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19609640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f141422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19e0a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14143d70 0x7f6f14145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.987 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19609640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f141422c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:40272/0 (socket says 172.21.15.161:40272) 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19609640 1 -- 172.21.15.161:0/991367822 learned_addr learned my addr 172.21.15.161:0/991367822 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f18e08640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f140736b0 0x7f6f14144650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19e0a640 1 -- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 msgr2=0x7f6f141422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19e0a640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f141422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19e0a640 1 -- 172.21.15.161:0/991367822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f140736b0 msgr2=0x7f6f14144650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19e0a640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f140736b0 0x7f6f14144650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19e0a640 1 -- 172.21.15.161:0/991367822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04009420 con 0x7f6f14143d70 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19609640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f141422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f18e08640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f140736b0 0x7f6f14144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f19e0a640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14143d70 0x7f6f14145120 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000be20 tx=0x7f6f1000bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f09ffb640 1 -- 172.21.15.161:0/991367822 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f1000ce70 con 0x7f6f14143d70 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f09ffb640 1 -- 172.21.15.161:0/991367822 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f10014440 con 0x7f6f14143d70 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f141df250 con 0x7f6f14143d70 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f09ffb640 1 -- 172.21.15.161:0/991367822 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f10014810 con 0x7f6f14143d70 2023-12-16T21:42:16.988 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f14142e70 con 0x7f6f14143d70 2023-12-16T21:42:16.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f09ffb640 1 -- 172.21.15.161:0/991367822 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6f10014c70 con 0x7f6f14143d70 2023-12-16T21:42:16.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f09ffb640 1 -- 172.21.15.161:0/991367822 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6f1001d6f0 con 0x7f6f14143d70 2023-12-16T21:42:16.989 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.982+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f141dedd0 con 0x7f6f14143d70 2023-12-16T21:42:16.993 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:16.990+0000 7f6f09ffb640 1 -- 172.21.15.161:0/991367822 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f6f141dedd0 con 0x7f6f14143d70 2023-12-16T21:42:17.060 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.058+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a6d39371-17a2-4ed3-90aa-eff61ee76362", "id": 0} v 0) v1 -- 0x7f6f14068960 con 0x7f6f14143d70 2023-12-16T21:42:17.071 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.066+0000 7f6f09ffb640 1 -- 172.21.15.161:0/991367822 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a6d39371-17a2-4ed3-90aa-eff61ee76362", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6f10019070 con 0x7f6f14143d70 2023-12-16T21:42:17.071 INFO:teuthology.orchestra.run.smithi161.stdout:0 2023-12-16T21:42:17.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14143d70 msgr2=0x7f6f14145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.072 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14143d70 0x7f6f14145120 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000be20 tx=0x7f6f1000bf10 comp rx=0 tx=0).stop 2023-12-16T21:42:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 shutdown_connections 2023-12-16T21:42:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6f14143d70 0x7f6f14145120 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6f140736b0 0x7f6f14144650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 --2- 172.21.15.161:0/991367822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6f1406b620 0x7f6f141422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 >> 172.21.15.161:0/991367822 conn(0x7f6f1406a580 msgr2=0x7f6f140b6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 shutdown_connections 2023-12-16T21:42:17.073 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.070+0000 7f6f1b86d640 1 -- 172.21.15.161:0/991367822 wait complete. 2023-12-16T21:42:17.087 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new a0fd9f81-9b98-46c5-b4e6-82adbcfc1064 1 2023-12-16T21:42:17.180 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:17.172+0000 7ff405085180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-16T21:42:17.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f8516a8f640 1 Processor -- start 2023-12-16T21:42:17.201 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f8516a8f640 1 -- start start 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f8516a8f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8510073710 0x7f85101483e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f8516a8f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510071240 0x7f851013e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f8516a8f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8510073af0 con 0x7f851013ea00 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f8516a8f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8510145490 con 0x7f8510073710 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f8516a8f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510147be0 con 0x7f8510071240 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f851482b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8510073710 0x7f85101483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f850ffff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510071240 0x7f851013e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f851482b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8510073710 0x7f85101483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47198/0 (socket says 172.21.15.161:47198) 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f851482b640 1 -- 172.21.15.161:0/32200641 learned_addr learned my addr 172.21.15.161:0/32200641 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f850ffff640 1 --2- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510071240 0x7f851013e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:40288/0 (socket says 172.21.15.161:40288) 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f851482b640 1 -- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510071240 msgr2=0x7f851013e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:17.203 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f851482b640 1 --2- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510071240 0x7f851013e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f851482b640 1 -- 172.21.15.161:0/32200641 >> v1:172.21.15.46:6789/0 conn(0x7f851013ea00 legacy=0x7f8510144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f851482b640 1 -- 172.21.15.161:0/32200641 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8510147ec0 con 0x7f8510073710 2023-12-16T21:42:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.198+0000 7f850ffff640 1 --2- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510071240 0x7f851013e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:17.204 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f851482b640 1 --2- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8510073710 0x7f85101483e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8500009870 tx=0x7f850001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd35b2b72c26aed server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f850f7fe640 1 -- 172.21.15.161:0/32200641 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000ab600 con 0x7f8510073710 2023-12-16T21:42:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f850f7fe640 1 -- 172.21.15.161:0/32200641 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85000a7050 con 0x7f8510073710 2023-12-16T21:42:17.205 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f850f7fe640 1 -- 172.21.15.161:0/32200641 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85000ab980 con 0x7f8510073710 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8510073710 msgr2=0x7f85101483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 --2- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8510073710 0x7f85101483e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8500009870 tx=0x7f850001c470 comp rx=0 tx=0).stop 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- 172.21.15.161:0/32200641 shutdown_connections 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 --2- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510071240 0x7f851013e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 --2- 172.21.15.161:0/32200641 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8510073710 0x7f85101483e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- 172.21.15.161:0/32200641 >> 172.21.15.161:0/32200641 conn(0x7f851006a580 msgr2=0x7f851006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- 172.21.15.161:0/32200641 shutdown_connections 2023-12-16T21:42:17.206 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- 172.21.15.161:0/32200641 wait complete. 2023-12-16T21:42:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 Processor -- start 2023-12-16T21:42:17.208 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- start start 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8510071240 0x7f851014a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510073710 0x7f85101416b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851013ea00 0x7f8510149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510141bf0 con 0x7f8510071240 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851014ab10 con 0x7f851013ea00 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f8516a8f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8510148b70 con 0x7f8510073710 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f850ffff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510073710 0x7f85101416b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f851502c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851013ea00 0x7f8510149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f850ffff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510073710 0x7f85101416b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:40296/0 (socket says 172.21.15.161:40296) 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f850ffff640 1 -- 172.21.15.161:0/474161513 learned_addr learned my addr 172.21.15.161:0/474161513 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f851502c640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851013ea00 0x7f8510149780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:47208/0 (socket says 172.21.15.161:47208) 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.202+0000 7f851482b640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8510071240 0x7f851014a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f851502c640 1 -- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510073710 msgr2=0x7f85101416b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.209 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f851502c640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510073710 0x7f85101416b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f851502c640 1 -- 172.21.15.161:0/474161513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8510071240 msgr2=0x7f851014a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f851502c640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8510071240 0x7f851014a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f851502c640 1 -- 172.21.15.161:0/474161513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009420 con 0x7f851013ea00 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f850ffff640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510073710 0x7f85101416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f851502c640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851013ea00 0x7f8510149780 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f850800be00 tx=0x7f850800bef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f850d7fa640 1 -- 172.21.15.161:0/474161513 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850800ce70 con 0x7f851013ea00 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f850d7fa640 1 -- 172.21.15.161:0/474161513 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8508003980 con 0x7f851013ea00 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f850d7fa640 1 -- 172.21.15.161:0/474161513 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508003d70 con 0x7f851013ea00 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85000094d0 con 0x7f851013ea00 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8510141e50 con 0x7f851013ea00 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f850d7fa640 1 -- 172.21.15.161:0/474161513 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f85080041f0 con 0x7f851013ea00 2023-12-16T21:42:17.210 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f850d7fa640 1 -- 172.21.15.161:0/474161513 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f8508014e10 con 0x7f851013ea00 2023-12-16T21:42:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8510068960 con 0x7f851013ea00 2023-12-16T21:42:17.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.206+0000 7f850d7fa640 1 -- 172.21.15.161:0/474161513 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f8508022060 con 0x7f851013ea00 2023-12-16T21:42:17.253 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.250+0000 7f623559d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-16T21:42:17.278 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.274+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a0fd9f81-9b98-46c5-b4e6-82adbcfc1064", "id": 1} v 0) v1 -- 0x7f8510149430 con 0x7f851013ea00 2023-12-16T21:42:17.288 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f850d7fa640 1 -- 172.21.15.161:0/474161513 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a0fd9f81-9b98-46c5-b4e6-82adbcfc1064", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8508014440 con 0x7f851013ea00 2023-12-16T21:42:17.288 INFO:teuthology.orchestra.run.smithi161.stdout:1 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851013ea00 msgr2=0x7f8510149780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851013ea00 0x7f8510149780 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f850800be00 tx=0x7f850800bef0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 shutdown_connections 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f851013ea00 0x7f8510149780 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8510073710 0x7f85101416b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 --2- 172.21.15.161:0/474161513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8510071240 0x7f851014a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 >> 172.21.15.161:0/474161513 conn(0x7f851006a580 msgr2=0x7f85100b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 shutdown_connections 2023-12-16T21:42:17.290 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.286+0000 7f8516a8f640 1 -- 172.21.15.161:0/474161513 wait complete. 2023-12-16T21:42:17.305 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 63203227-89d5-467b-a0ef-6e66270c12a2 2 2023-12-16T21:42:17.309 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.306+0000 7f623559d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-16T21:42:17.364 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:17.356+0000 7ff405085180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-16T21:42:17.364 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.362+0000 7f623559d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-16T21:42:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.410+0000 7f71d0ee7640 1 Processor -- start 2023-12-16T21:42:17.415 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.410+0000 7f71d0ee7640 1 -- start start 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc071ca0 0x7f71cc0b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc0b3f50 con 0x7f71cc148550 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71cc06d810 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc0b3220 con 0x7f71cc071ca0 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71ca59c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc071ca0 0x7f71cc0b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71c9d9b640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f71cc0b2dc0 0x7f71cc147e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:42348/0 (socket says 172.21.15.161:42348) 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71c9d9b640 1 -- 172.21.15.161:0/1775173513 learned_addr learned my addr 172.21.15.161:0/1775173513 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71c959a640 1 -- 172.21.15.161:0/1775173513 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027340604 0 0) 0x7f71cc06d810 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.416 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71ca59c640 1 -- 172.21.15.161:0/1775173513 >> v1:172.21.15.161:6789/0 conn(0x7f71cc0b2dc0 legacy=0x7f71cc147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71ca59c640 1 -- 172.21.15.161:0/1775173513 >> v1:172.21.15.46:6789/0 conn(0x7f71cc148550 legacy=0x7f71cc14ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:17.417 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71ca59c640 1 -- 172.21.15.161:0/1775173513 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc072080 con 0x7f71cc071ca0 2023-12-16T21:42:17.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71ca59c640 1 --2- 172.21.15.161:0/1775173513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc071ca0 0x7f71cc0b2880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097f0 tx=0x7f71bc01c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88748cec95d01960 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71c959a640 1 -- 172.21.15.161:0/1775173513 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0ab600 con 0x7f71cc071ca0 2023-12-16T21:42:17.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71c959a640 1 -- 172.21.15.161:0/1775173513 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc0a7050 con 0x7f71cc071ca0 2023-12-16T21:42:17.418 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71c959a640 1 -- 172.21.15.161:0/1775173513 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0ab980 con 0x7f71cc071ca0 2023-12-16T21:42:17.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- 172.21.15.161:0/1775173513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc071ca0 msgr2=0x7f71cc0b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 --2- 172.21.15.161:0/1775173513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc071ca0 0x7f71cc0b2880 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f71bc0097f0 tx=0x7f71bc01c3f0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- 172.21.15.161:0/1775173513 shutdown_connections 2023-12-16T21:42:17.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 --2- 172.21.15.161:0/1775173513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc071ca0 0x7f71cc0b2880 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.419 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- 172.21.15.161:0/1775173513 >> 172.21.15.161:0/1775173513 conn(0x7f71cc0b90b0 msgr2=0x7f71cc0b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- 172.21.15.161:0/1775173513 shutdown_connections 2023-12-16T21:42:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.414+0000 7f71d0ee7640 1 -- 172.21.15.161:0/1775173513 wait complete. 2023-12-16T21:42:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 Processor -- start 2023-12-16T21:42:17.420 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 -- start start 2023-12-16T21:42:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71cc071ca0 0x7f71cc1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc0b2dc0 0x7f71cc1de280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71cc148550 0x7f71cc1de7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc1442f0 con 0x7f71cc071ca0 2023-12-16T21:42:17.422 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc143450 con 0x7f71cc148550 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71cc1447d0 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc0b2dc0 0x7f71cc1de280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71cad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71cc148550 0x7f71cc1de7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc0b2dc0 0x7f71cc1de280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:40322/0 (socket says 172.21.15.161:40322) 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 -- 172.21.15.161:0/3573368757 learned_addr learned my addr 172.21.15.161:0/3573368757 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71ca59c640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71cc071ca0 0x7f71cc1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 -- 172.21.15.161:0/3573368757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71cc148550 msgr2=0x7f71cc1de7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71cc148550 0x7f71cc1de7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 -- 172.21.15.161:0/3573368757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71cc071ca0 msgr2=0x7f71cc1423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71cc071ca0 0x7f71cc1423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 -- 172.21.15.161:0/3573368757 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc009420 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71ca59c640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71cc071ca0 0x7f71cc1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71c9d9b640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc0b2dc0 0x7f71cc1de280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c520 tx=0x7f71c000c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.423 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71baffd640 1 -- 172.21.15.161:0/3573368757 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00074e0 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71baffd640 1 -- 172.21.15.161:0/3573368757 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c0007640 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71cc1444f0 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71baffd640 1 -- 172.21.15.161:0/3573368757 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00049f0 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc143940 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71baffd640 1 -- 172.21.15.161:0/3573368757 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f71c0012400 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71baffd640 1 -- 172.21.15.161:0/3573368757 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f71c001c810 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.424 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.418+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71cc067f60 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.425 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.422+0000 7f71baffd640 1 -- 172.21.15.161:0/3573368757 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f71c0021060 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.446 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.442+0000 7f623559d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-16T21:42:17.492 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.490+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "63203227-89d5-467b-a0ef-6e66270c12a2", "id": 2} v 0) v1 -- 0x7f71cc0b5120 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.499 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71baffd640 1 -- 172.21.15.161:0/3573368757 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "63203227-89d5-467b-a0ef-6e66270c12a2", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f71c0018070 con 0x7f71cc0b2dc0 2023-12-16T21:42:17.500 INFO:teuthology.orchestra.run.smithi161.stdout:2 2023-12-16T21:42:17.500 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f623559d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc0b2dc0 msgr2=0x7f71cc1de280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc0b2dc0 0x7f71cc1de280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f71c000c520 tx=0x7f71c000c9e0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 shutdown_connections 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f71cc148550 0x7f71cc1de7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f71cc0b2dc0 0x7f71cc1de280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 --2- 172.21.15.161:0/3573368757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71cc071ca0 0x7f71cc1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 >> 172.21.15.161:0/3573368757 conn(0x7f71cc0b90b0 msgr2=0x7f71cc0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 shutdown_connections 2023-12-16T21:42:17.501 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.498+0000 7f71d0ee7640 1 -- 172.21.15.161:0/3573368757 wait complete. 2023-12-16T21:42:17.516 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 8225ce9b-2382-468e-8aaf-2dc5c7807217 3 2023-12-16T21:42:17.581 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.578+0000 7f623559d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-16T21:42:17.637 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.634+0000 7f623559d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-16T21:42:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe85b444640 1 Processor -- start 2023-12-16T21:42:17.682 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe85b444640 1 -- start start 2023-12-16T21:42:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe85b444640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe85406b300 0x7fe8540b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.683 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe85b444640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe8540728e0 0x7fe854146920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe85b444640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8540b1900 con 0x7fe854143040 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe85b444640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe854071fb0 con 0x7fe8540728e0 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe85b444640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85406b6e0 con 0x7fe85406b300 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8591e0640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe85406b300 0x7fe8540b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8591e0640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe85406b300 0x7fe8540b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:42728/0 (socket says 172.21.15.161:42728) 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8589df640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe8540728e0 0x7fe854146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8591e0640 1 -- 172.21.15.161:0/1811597340 learned_addr learned my addr 172.21.15.161:0/1811597340 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8589df640 1 -- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe85406b300 msgr2=0x7fe8540b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8589df640 1 --2- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe85406b300 0x7fe8540b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8589df640 1 -- 172.21.15.161:0/1811597340 >> v1:172.21.15.161:6789/0 conn(0x7fe854143040 legacy=0x7fe8541434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8589df640 1 -- 172.21.15.161:0/1811597340 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8540b2750 con 0x7fe8540728e0 2023-12-16T21:42:17.684 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.678+0000 7fe8591e0640 1 --2- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe85406b300 0x7fe8540b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe8589df640 1 --2- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8540728e0 0x7fe854146920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe8440097c0 tx=0x7fe84401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30cff6b6879a305 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe84bfff640 1 -- 172.21.15.161:0/1811597340 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440ab600 con 0x7fe8540728e0 2023-12-16T21:42:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe84bfff640 1 -- 172.21.15.161:0/1811597340 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8440a7050 con 0x7fe8540728e0 2023-12-16T21:42:17.685 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe84bfff640 1 -- 172.21.15.161:0/1811597340 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8440ab980 con 0x7fe8540728e0 2023-12-16T21:42:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 -- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8540728e0 msgr2=0x7fe854146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 --2- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8540728e0 0x7fe854146920 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe8440097c0 tx=0x7fe84401c3f0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 -- 172.21.15.161:0/1811597340 shutdown_connections 2023-12-16T21:42:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 --2- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8540728e0 0x7fe854146920 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 --2- 172.21.15.161:0/1811597340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe85406b300 0x7fe8540b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.686 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 -- 172.21.15.161:0/1811597340 >> 172.21.15.161:0/1811597340 conn(0x7fe85406a580 msgr2=0x7fe85406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 -- 172.21.15.161:0/1811597340 shutdown_connections 2023-12-16T21:42:17.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 -- 172.21.15.161:0/1811597340 wait complete. 2023-12-16T21:42:17.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 Processor -- start 2023-12-16T21:42:17.687 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.682+0000 7fe85b444640 1 -- start start 2023-12-16T21:42:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85406b300 0x7fe854070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8540728e0 0x7fe854070960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.690 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854143040 0x7fe8541e2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8541313f0 con 0x7fe8540728e0 2023-12-16T21:42:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe854136010 con 0x7fe85406b300 2023-12-16T21:42:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8540b6430 con 0x7fe854143040 2023-12-16T21:42:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85406b300 0x7fe854070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85406b300 0x7fe854070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34790/0 (socket says 172.21.15.161:34790) 2023-12-16T21:42:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 -- 172.21.15.161:0/2100061316 learned_addr learned my addr 172.21.15.161:0/2100061316 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:17.691 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8599e1640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854143040 0x7fe8541e2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 -- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854143040 msgr2=0x7fe8541e2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854143040 0x7fe8541e2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8589df640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8540728e0 0x7fe854070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 -- 172.21.15.161:0/2100061316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8540728e0 msgr2=0x7fe854070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8540728e0 0x7fe854070960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.692 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 -- 172.21.15.161:0/2100061316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c00bd40 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe8591e0640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85406b300 0x7fe854070420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009e20 tx=0x7fe84c00e490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe849ffb640 1 -- 172.21.15.161:0/2100061316 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c010040 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe849ffb640 1 -- 172.21.15.161:0/2100061316 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84c00ee00 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe844009420 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe849ffb640 1 -- 172.21.15.161:0/2100061316 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe84c015680 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe854136710 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe849ffb640 1 -- 172.21.15.161:0/2100061316 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe84c015ae0 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe849ffb640 1 -- 172.21.15.161:0/2100061316 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe84c015d90 con 0x7fe85406b300 2023-12-16T21:42:17.693 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.686+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8540b1c80 con 0x7fe85406b300 2023-12-16T21:42:17.694 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.690+0000 7fe849ffb640 1 -- 172.21.15.161:0/2100061316 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7fe84c01a070 con 0x7fe85406b300 2023-12-16T21:42:17.760 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.758+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8225ce9b-2382-468e-8aaf-2dc5c7807217", "id": 3} v 0) v1 -- 0x7fe854139e90 con 0x7fe85406b300 2023-12-16T21:42:17.765 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe849ffb640 1 -- 172.21.15.161:0/2100061316 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8225ce9b-2382-468e-8aaf-2dc5c7807217", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe854139e90 con 0x7fe85406b300 2023-12-16T21:42:17.766 INFO:teuthology.orchestra.run.smithi161.stdout:3 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85406b300 msgr2=0x7fe854070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85406b300 0x7fe854070420 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe84c009e20 tx=0x7fe84c00e490 comp rx=0 tx=0).stop 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 shutdown_connections 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe854143040 0x7fe8541e2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8540728e0 0x7fe854070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 --2- 172.21.15.161:0/2100061316 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe85406b300 0x7fe854070420 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 >> 172.21.15.161:0/2100061316 conn(0x7fe85406a580 msgr2=0x7fe854142230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 shutdown_connections 2023-12-16T21:42:17.767 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.762+0000 7fe85b444640 1 -- 172.21.15.161:0/2100061316 wait complete. 2023-12-16T21:42:17.782 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 1c1d186f-c49d-4a50-986f-c4a20dee5f0d 4 2023-12-16T21:42:17.784 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.782+0000 7f623559d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-16T21:42:17.847 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:17.842+0000 7f623559d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-16T21:42:17.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f96412b0640 1 Processor -- start 2023-12-16T21:42:17.897 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f96412b0640 1 -- start start 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f96412b0640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f963c0ba920 0x7f963c140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f96412b0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963c0b5d80 con 0x7f963c06d000 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f96412b0640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963c0bad00 con 0x7f963c0b52e0 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f96412b0640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963c0b6020 con 0x7f963c0ba920 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f963affd640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f963c0ba920 0x7f963c140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f963a7fc640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f963c0b52e0 0x7f963c06c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33356/0 (socket says 172.21.15.161:33356) 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f963affd640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f963c0ba920 0x7f963c140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34802/0 (socket says 172.21.15.161:34802) 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f963a7fc640 1 -- 172.21.15.161:0/62585863 learned_addr learned my addr 172.21.15.161:0/62585863 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f963affd640 1 -- 172.21.15.161:0/62585863 >> v1:172.21.15.161:6789/0 conn(0x7f963c0b52e0 legacy=0x7f963c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.899 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f963affd640 1 -- 172.21.15.161:0/62585863 >> v1:172.21.15.46:6789/0 conn(0x7f963c06d000 legacy=0x7f963c151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:17.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f9639ffb640 1 -- 172.21.15.161:0/62585863 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809805615 0 0) 0x7f963c0bad00 con 0x7f963c0b52e0 2023-12-16T21:42:17.900 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.894+0000 7f963affd640 1 -- 172.21.15.161:0/62585863 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c0b6300 con 0x7f963c0ba920 2023-12-16T21:42:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f963affd640 1 --2- 172.21.15.161:0/62585863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c0ba920 0x7f963c140580 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96240097c0 tx=0x7f962401c320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb8d39731b3bd586 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f9639ffb640 1 -- 172.21.15.161:0/62585863 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240ab600 con 0x7f963c0ba920 2023-12-16T21:42:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f9639ffb640 1 -- 172.21.15.161:0/62585863 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96240a7050 con 0x7f963c0ba920 2023-12-16T21:42:17.902 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f9639ffb640 1 -- 172.21.15.161:0/62585863 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240ab980 con 0x7f963c0ba920 2023-12-16T21:42:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- 172.21.15.161:0/62585863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c0ba920 msgr2=0x7f963c140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 --2- 172.21.15.161:0/62585863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c0ba920 0x7f963c140580 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f96240097c0 tx=0x7f962401c320 comp rx=0 tx=0).stop 2023-12-16T21:42:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- 172.21.15.161:0/62585863 shutdown_connections 2023-12-16T21:42:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 --2- 172.21.15.161:0/62585863 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c0ba920 0x7f963c140580 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- 172.21.15.161:0/62585863 >> 172.21.15.161:0/62585863 conn(0x7f963c06ab80 msgr2=0x7f963c0b9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.903 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- 172.21.15.161:0/62585863 shutdown_connections 2023-12-16T21:42:17.904 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- 172.21.15.161:0/62585863 wait complete. 2023-12-16T21:42:17.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 Processor -- start 2023-12-16T21:42:17.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- start start 2023-12-16T21:42:17.906 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c06d000 0x7f963c0b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f963c0b52e0 0x7f963c13dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0ba920 0x7f963c13e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c13e6c0 con 0x7f963c0ba920 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c0b2940 con 0x7f963c06d000 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.898+0000 7f96412b0640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963c1e2510 con 0x7f963c0b52e0 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c06d000 0x7f963c0b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963a7fc640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f963c0b52e0 0x7f963c13dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c06d000 0x7f963c0b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34816/0 (socket says 172.21.15.161:34816) 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 -- 172.21.15.161:0/3052355715 learned_addr learned my addr 172.21.15.161:0/3052355715 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963b7fe640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0ba920 0x7f963c13e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 -- 172.21.15.161:0/3052355715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f963c0b52e0 msgr2=0x7f963c13dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f963c0b52e0 0x7f963c13dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 -- 172.21.15.161:0/3052355715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0ba920 msgr2=0x7f963c13e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0ba920 0x7f963c13e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.907 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 -- 172.21.15.161:0/3052355715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9624009420 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963b7fe640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0ba920 0x7f963c13e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f963affd640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c06d000 0x7f963c0b2400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96240ae040 tx=0x7f962401ab60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96337fe640 1 -- 172.21.15.161:0/3052355715 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240b7650 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96337fe640 1 -- 172.21.15.161:0/3052355715 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96240b77d0 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96337fe640 1 -- 172.21.15.161:0/3052355715 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96240b7b80 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c1e27f0 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c13f330 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96337fe640 1 -- 172.21.15.161:0/3052355715 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f96240c0400 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96337fe640 1 -- 172.21.15.161:0/3052355715 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f96240b5910 con 0x7f963c06d000 2023-12-16T21:42:17.908 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.902+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963c068960 con 0x7f963c06d000 2023-12-16T21:42:17.909 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.906+0000 7f96337fe640 1 -- 172.21.15.161:0/3052355715 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f96240decf0 con 0x7f963c06d000 2023-12-16T21:42:17.955 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:17.948+0000 7ff405085180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-16T21:42:17.974 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.970+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1c1d186f-c49d-4a50-986f-c4a20dee5f0d", "id": 4} v 0) v1 -- 0x7f963c1e2ca0 con 0x7f963c06d000 2023-12-16T21:42:17.979 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96337fe640 1 -- 172.21.15.161:0/3052355715 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1c1d186f-c49d-4a50-986f-c4a20dee5f0d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f96240cd030 con 0x7f963c06d000 2023-12-16T21:42:17.980 INFO:teuthology.orchestra.run.smithi161.stdout:4 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c06d000 msgr2=0x7f963c0b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c06d000 0x7f963c0b2400 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f96240ae040 tx=0x7f962401ab60 comp rx=0 tx=0).stop 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 shutdown_connections 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f963c0ba920 0x7f963c13e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f963c0b52e0 0x7f963c13dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 --2- 172.21.15.161:0/3052355715 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f963c06d000 0x7f963c0b2400 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 >> 172.21.15.161:0/3052355715 conn(0x7f963c06ab80 msgr2=0x7f963c0b9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 shutdown_connections 2023-12-16T21:42:17.981 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:17.978+0000 7f96412b0640 1 -- 172.21.15.161:0/3052355715 wait complete. 2023-12-16T21:42:17.996 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new a7e2dcfb-a4c2-4e55-90a4-79e4590292b0 5 2023-12-16T21:42:18.015 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.008+0000 7ff405085180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-16T21:42:18.071 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.064+0000 7ff405085180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-16T21:42:18.105 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f97bec640 1 Processor -- start 2023-12-16T21:42:18.106 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f97bec640 1 -- start start 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f97bec640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f9006cdc0 0x7f9f9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f97bec640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9f900b5330 0x7f9f901405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f97bec640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f900afb30 con 0x7f9f900b40b0 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f97bec640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f900704b0 con 0x7f9f900b5330 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f97bec640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9006f8c0 con 0x7f9f9006cdc0 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f95988640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f9006cdc0 0x7f9f9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f95988640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f9006cdc0 0x7f9f9006fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:42766/0 (socket says 172.21.15.161:42766) 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f95988640 1 -- 172.21.15.161:0/840532304 learned_addr learned my addr 172.21.15.161:0/840532304 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f96189640 1 --2- 172.21.15.161:0/840532304 >> v2:172.21.15.46:3300/0 conn(0x7f9f900b5330 0x7f9f901405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f95988640 1 -- 172.21.15.161:0/840532304 >> v2:172.21.15.46:3300/0 conn(0x7f9f900b5330 msgr2=0x7f9f901405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:18.107 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f95988640 1 --2- 172.21.15.161:0/840532304 >> v2:172.21.15.46:3300/0 conn(0x7f9f900b5330 0x7f9f901405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f95988640 1 -- 172.21.15.161:0/840532304 >> v1:172.21.15.46:6789/0 conn(0x7f9f900b40b0 legacy=0x7f9f90071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.108 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.102+0000 7f9f95988640 1 -- 172.21.15.161:0/840532304 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9006d1a0 con 0x7f9f9006cdc0 2023-12-16T21:42:18.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f95988640 1 --2- 172.21.15.161:0/840532304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f9006cdc0 0x7f9f9006fd50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009870 tx=0x7f9f8001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3178313d4df16a52 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f94986640 1 -- 172.21.15.161:0/840532304 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800ad640 con 0x7f9f9006cdc0 2023-12-16T21:42:18.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f94986640 1 -- 172.21.15.161:0/840532304 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f800ad7a0 con 0x7f9f9006cdc0 2023-12-16T21:42:18.109 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f94986640 1 -- 172.21.15.161:0/840532304 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800adb30 con 0x7f9f9006cdc0 2023-12-16T21:42:18.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 -- 172.21.15.161:0/840532304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f9006cdc0 msgr2=0x7f9f9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 --2- 172.21.15.161:0/840532304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f9006cdc0 0x7f9f9006fd50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f80009870 tx=0x7f9f8001c480 comp rx=0 tx=0).stop 2023-12-16T21:42:18.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 -- 172.21.15.161:0/840532304 shutdown_connections 2023-12-16T21:42:18.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 --2- 172.21.15.161:0/840532304 >> v2:172.21.15.46:3300/0 conn(0x7f9f900b5330 0x7f9f901405e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 --2- 172.21.15.161:0/840532304 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f9006cdc0 0x7f9f9006fd50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.110 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 -- 172.21.15.161:0/840532304 >> 172.21.15.161:0/840532304 conn(0x7f9f900b90b0 msgr2=0x7f9f900b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 -- 172.21.15.161:0/840532304 shutdown_connections 2023-12-16T21:42:18.111 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 -- 172.21.15.161:0/840532304 wait complete. 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 Processor -- start 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.106+0000 7f9f97bec640 1 -- start start 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f9006cdc0 0x7f9f901e2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f900b40b0 0x7f9f900affc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f900b5330 0x7f9f900b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f90145e40 con 0x7f9f900b5330 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f901e2b40 con 0x7f9f9006cdc0 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f901e32c0 con 0x7f9f900b40b0 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f9006cdc0 0x7f9f901e2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95187640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f900b40b0 0x7f9f900affc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.113 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f9006cdc0 0x7f9f901e2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34832/0 (socket says 172.21.15.161:34832) 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 -- 172.21.15.161:0/3363511369 learned_addr learned my addr 172.21.15.161:0/3363511369 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f96189640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f900b5330 0x7f9f900b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 -- 172.21.15.161:0/3363511369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f900b40b0 msgr2=0x7f9f900affc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f900b40b0 0x7f9f900affc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 -- 172.21.15.161:0/3363511369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f900b5330 msgr2=0x7f9f900b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f900b5330 0x7f9f900b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 -- 172.21.15.161:0/3363511369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f800094d0 con 0x7f9f9006cdc0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f95988640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f9006cdc0 0x7f9f901e2600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b0040 tx=0x7f9f80019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f867fc640 1 -- 172.21.15.161:0/3363511369 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800b6640 con 0x7f9f9006cdc0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f867fc640 1 -- 172.21.15.161:0/3363511369 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f800a6040 con 0x7f9f9006cdc0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f900b2750 con 0x7f9f9006cdc0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f867fc640 1 -- 172.21.15.161:0/3363511369 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f800b6a00 con 0x7f9f9006cdc0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f901e2da0 con 0x7f9f9006cdc0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f867fc640 1 -- 172.21.15.161:0/3363511369 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9f800b4070 con 0x7f9f9006cdc0 2023-12-16T21:42:18.114 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f867fc640 1 -- 172.21.15.161:0/3363511369 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9f800bfb30 con 0x7f9f9006cdc0 2023-12-16T21:42:18.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.110+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f90067f60 con 0x7f9f9006cdc0 2023-12-16T21:42:18.115 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.114+0000 7f9f867fc640 1 -- 172.21.15.161:0/3363511369 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f9f800ba020 con 0x7f9f9006cdc0 2023-12-16T21:42:18.152 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.144+0000 7ff405085180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-16T21:42:18.206 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.200+0000 7ff405085180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-16T21:42:18.211 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.210+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a7e2dcfb-a4c2-4e55-90a4-79e4590292b0", "id": 5} v 0) v1 -- 0x7f9f90143b70 con 0x7f9f9006cdc0 2023-12-16T21:42:18.218 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.214+0000 7f9f867fc640 1 -- 172.21.15.161:0/3363511369 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7e2dcfb-a4c2-4e55-90a4-79e4590292b0", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9f90143b70 con 0x7f9f9006cdc0 2023-12-16T21:42:18.219 INFO:teuthology.orchestra.run.smithi161.stdout:5 2023-12-16T21:42:18.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f9006cdc0 msgr2=0x7f9f901e2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f9006cdc0 0x7f9f901e2600 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9f800b0040 tx=0x7f9f80019220 comp rx=0 tx=0).stop 2023-12-16T21:42:18.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 shutdown_connections 2023-12-16T21:42:18.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9f900b5330 0x7f9f900b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.221 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9f900b40b0 0x7f9f900affc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 --2- 172.21.15.161:0/3363511369 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9f9006cdc0 0x7f9f901e2600 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 >> 172.21.15.161:0/3363511369 conn(0x7f9f900b90b0 msgr2=0x7f9f900b9490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 shutdown_connections 2023-12-16T21:42:18.222 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.218+0000 7f9f97bec640 1 -- 172.21.15.161:0/3363511369 wait complete. 2023-12-16T21:42:18.247 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new 0e507728-c7c8-4fe2-98c1-47c2315463a5 6 2023-12-16T21:42:18.287 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.280+0000 7ff405085180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-16T21:42:18.343 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.336+0000 7ff405085180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-16T21:42:18.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e50cce640 1 Processor -- start 2023-12-16T21:42:18.357 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e50cce640 1 -- start start 2023-12-16T21:42:18.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e50cce640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1e4c0b4020 0x7f1e4c071790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e50cce640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c0b4b90 con 0x7f1e4c071cd0 2023-12-16T21:42:18.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e50cce640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c0b6080 con 0x7f1e4c0b5ca0 2023-12-16T21:42:18.358 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e50cce640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c070e00 con 0x7f1e4c0b4020 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e4a59c640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1e4c0b5ca0 0x7f1e4c1405e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.161:33368/0 (socket says 172.21.15.161:33368) 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e4a59c640 1 -- 172.21.15.161:0/971531034 learned_addr learned my addr 172.21.15.161:0/971531034 (peer_addr_for_me v1:172.21.15.161:0/0) 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e49d9b640 1 --2- 172.21.15.161:0/971531034 >> v2:172.21.15.161:3300/0 conn(0x7f1e4c0b4020 0x7f1e4c071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e4959a640 1 -- 172.21.15.161:0/971531034 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285435091 0 0) 0x7f1e4c0b6080 con 0x7f1e4c0b5ca0 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e4959a640 1 -- 172.21.15.161:0/971531034 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e34003530 con 0x7f1e4c0b5ca0 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e49d9b640 1 -- 172.21.15.161:0/971531034 >> v1:172.21.15.161:6789/0 conn(0x7f1e4c0b5ca0 legacy=0x7f1e4c1405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e49d9b640 1 -- 172.21.15.161:0/971531034 >> v1:172.21.15.46:6789/0 conn(0x7f1e4c071cd0 legacy=0x7f1e4c150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:18.359 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.354+0000 7f1e49d9b640 1 -- 172.21.15.161:0/971531034 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c072120 con 0x7f1e4c0b4020 2023-12-16T21:42:18.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e49d9b640 1 --2- 172.21.15.161:0/971531034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b4020 0x7f1e4c071790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e4001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b67a71516ad5b server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4959a640 1 -- 172.21.15.161:0/971531034 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400ad680 con 0x7f1e4c0b4020 2023-12-16T21:42:18.360 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4959a640 1 -- 172.21.15.161:0/971531034 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e400ad7e0 con 0x7f1e4c0b4020 2023-12-16T21:42:18.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4959a640 1 -- 172.21.15.161:0/971531034 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e400adb70 con 0x7f1e4c0b4020 2023-12-16T21:42:18.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- 172.21.15.161:0/971531034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b4020 msgr2=0x7f1e4c071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 --2- 172.21.15.161:0/971531034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b4020 0x7f1e4c071790 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e400097c0 tx=0x7f1e4001c380 comp rx=0 tx=0).stop 2023-12-16T21:42:18.361 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- 172.21.15.161:0/971531034 shutdown_connections 2023-12-16T21:42:18.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 --2- 172.21.15.161:0/971531034 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b4020 0x7f1e4c071790 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- 172.21.15.161:0/971531034 >> 172.21.15.161:0/971531034 conn(0x7f1e4c06b860 msgr2=0x7f1e4c06bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- 172.21.15.161:0/971531034 shutdown_connections 2023-12-16T21:42:18.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- 172.21.15.161:0/971531034 wait complete. 2023-12-16T21:42:18.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 Processor -- start 2023-12-16T21:42:18.362 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- start start 2023-12-16T21:42:18.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e4c071cd0 0x7f1e4c146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e4c0b4020 0x7f1e4c141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.364 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b5ca0 0x7f1e4c1e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c132bd0 con 0x7f1e4c0b4020 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c141720 con 0x7f1e4c0b5ca0 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e50cce640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c1e24a0 con 0x7f1e4c071cd0 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b5ca0 0x7f1e4c1e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4a59c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e4c071cd0 0x7f1e4c146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4ad9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b5ca0 0x7f1e4c1e29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34848/0 (socket says 172.21.15.161:34848) 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4ad9d640 1 -- 172.21.15.161:0/456208125 learned_addr learned my addr 172.21.15.161:0/456208125 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e49d9b640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e4c0b4020 0x7f1e4c141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4a59c640 1 -- 172.21.15.161:0/456208125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b5ca0 msgr2=0x7f1e4c1e29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4a59c640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b5ca0 0x7f1e4c1e29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4a59c640 1 -- 172.21.15.161:0/456208125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e4c0b4020 msgr2=0x7f1e4c141050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4a59c640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e4c0b4020 0x7f1e4c141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.365 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.358+0000 7f1e4a59c640 1 -- 172.21.15.161:0/456208125 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40009420 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e4a59c640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e4c071cd0 0x7f1e4c146b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c390 tx=0x7f1e3c00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e3affd640 1 -- 172.21.15.161:0/456208125 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c005cf0 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e3affd640 1 -- 172.21.15.161:0/456208125 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c005e50 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c142840 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e3affd640 1 -- 172.21.15.161:0/456208125 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c0072d0 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c14a830 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e3affd640 1 -- 172.21.15.161:0/456208125 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1e3c01c440 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e3affd640 1 -- 172.21.15.161:0/456208125 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f1e3c0129f0 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c141590 con 0x7f1e4c071cd0 2023-12-16T21:42:18.366 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.362+0000 7f1e3affd640 1 -- 172.21.15.161:0/456208125 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f1e3c021060 con 0x7f1e4c071cd0 2023-12-16T21:42:18.432 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.430+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0e507728-c7c8-4fe2-98c1-47c2315463a5", "id": 6} v 0) v1 -- 0x7f1e4c067f20 con 0x7f1e4c071cd0 2023-12-16T21:42:18.439 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.434+0000 7f1e3affd640 1 -- 172.21.15.161:0/456208125 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e507728-c7c8-4fe2-98c1-47c2315463a5", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1e3c017070 con 0x7f1e4c071cd0 2023-12-16T21:42:18.439 INFO:teuthology.orchestra.run.smithi161.stdout:6 2023-12-16T21:42:18.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e4c071cd0 msgr2=0x7f1e4c146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e4c071cd0 0x7f1e4c146b60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00c390 tx=0x7f1e3c00c850 comp rx=0 tx=0).stop 2023-12-16T21:42:18.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 shutdown_connections 2023-12-16T21:42:18.440 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1e4c0b5ca0 0x7f1e4c1e29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1e4c0b4020 0x7f1e4c141050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 --2- 172.21.15.161:0/456208125 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1e4c071cd0 0x7f1e4c146b60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 >> 172.21.15.161:0/456208125 conn(0x7f1e4c06b860 msgr2=0x7f1e4c0adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 shutdown_connections 2023-12-16T21:42:18.441 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.438+0000 7f1e50cce640 1 -- 172.21.15.161:0/456208125 wait complete. 2023-12-16T21:42:18.455 DEBUG:teuthology.orchestra.run.smithi161:> sudo ceph --cluster ceph osd new bccb177c-3eb8-44e6-9694-0b3c11292cff 7 2023-12-16T21:42:18.469 INFO:tasks.ceph.mgr.x.smithi161.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-16T21:42:18.470 INFO:tasks.ceph.mgr.x.smithi161.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-16T21:42:18.470 INFO:tasks.ceph.mgr.x.smithi161.stderr: from numpy import show_config as show_numpy_config 2023-12-16T21:42:18.489 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.484+0000 7ff405085180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-16T21:42:18.506 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:18.502+0000 7f623559d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-16T21:42:18.551 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:18.544+0000 7ff405085180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-16T21:42:18.561 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 Processor -- start 2023-12-16T21:42:18.562 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 -- start start 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700ba920 0x7f7770140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f77700b52e0 0x7f7770140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f777006c8f0 0x7f777006cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77700b5d80 con 0x7f777006c8f0 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f777006d280 con 0x7f77700b52e0 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f7774cfe640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77700bad00 con 0x7f77700ba920 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700ba920 0x7f7770140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776dd9b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f77700b52e0 0x7f7770140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700ba920 0x7f7770140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.161:42782/0 (socket says 172.21.15.161:42782) 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 -- 172.21.15.161:0/2230519643 learned_addr learned my addr 172.21.15.161:0/2230519643 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:18.563 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776ed9d640 1 --2- 172.21.15.161:0/2230519643 >> v2:172.21.15.46:3300/0 conn(0x7f777006c8f0 0x7f777006cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 -- 172.21.15.161:0/2230519643 >> v2:172.21.15.161:3300/0 conn(0x7f77700b52e0 msgr2=0x7f7770140ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:18.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 --2- 172.21.15.161:0/2230519643 >> v2:172.21.15.161:3300/0 conn(0x7f77700b52e0 0x7f7770140ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 -- 172.21.15.161:0/2230519643 >> v2:172.21.15.46:3300/0 conn(0x7f777006c8f0 msgr2=0x7f777006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:18.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 --2- 172.21.15.161:0/2230519643 >> v2:172.21.15.46:3300/0 conn(0x7f777006c8f0 0x7f777006cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.564 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.558+0000 7f776e59c640 1 -- 172.21.15.161:0/2230519643 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700b6020 con 0x7f77700ba920 2023-12-16T21:42:18.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f776e59c640 1 --2- 172.21.15.161:0/2230519643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700ba920 0x7f7770140580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f777006b410 tx=0x7f776001c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac624a362090e89 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f776d59a640 1 -- 172.21.15.161:0/2230519643 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600a5610 con 0x7f77700ba920 2023-12-16T21:42:18.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f776d59a640 1 -- 172.21.15.161:0/2230519643 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77600a5770 con 0x7f77700ba920 2023-12-16T21:42:18.565 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f776d59a640 1 -- 172.21.15.161:0/2230519643 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600a5b00 con 0x7f77700ba920 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 -- 172.21.15.161:0/2230519643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700ba920 msgr2=0x7f7770140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2230519643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700ba920 0x7f7770140580 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f777006b410 tx=0x7f776001c880 comp rx=0 tx=0).stop 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 -- 172.21.15.161:0/2230519643 shutdown_connections 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2230519643 >> v2:172.21.15.46:3300/0 conn(0x7f777006c8f0 0x7f777006cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2230519643 >> v2:172.21.15.161:3300/0 conn(0x7f77700b52e0 0x7f7770140ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2230519643 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700ba920 0x7f7770140580 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 -- 172.21.15.161:0/2230519643 >> 172.21.15.161:0/2230519643 conn(0x7f777006ab80 msgr2=0x7f77700b9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 -- 172.21.15.161:0/2230519643 shutdown_connections 2023-12-16T21:42:18.566 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.562+0000 7f7774cfe640 1 -- 172.21.15.161:0/2230519643 wait complete. 2023-12-16T21:42:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 Processor -- start 2023-12-16T21:42:18.567 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 -- start start 2023-12-16T21:42:18.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777006c8f0 0x7f7770146fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.570 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700b52e0 0x7f77700b2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77700ba920 0x7f77700b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7770146140 con 0x7f77700ba920 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77700b13b0 con 0x7f777006c8f0 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77700b32f0 con 0x7f77700b52e0 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777006c8f0 0x7f7770146fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776dd9b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700b52e0 0x7f77700b2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777006c8f0 0x7f7770146fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.161:34858/0 (socket says 172.21.15.161:34858) 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 -- 172.21.15.161:0/2528201712 learned_addr learned my addr 172.21.15.161:0/2528201712 (peer_addr_for_me v2:172.21.15.161:0/0) 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776ed9d640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77700ba920 0x7f77700b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.571 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 -- 172.21.15.161:0/2528201712 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700b52e0 msgr2=0x7f77700b2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700b52e0 0x7f77700b2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 -- 172.21.15.161:0/2528201712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77700ba920 msgr2=0x7f77700b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77700ba920 0x7f77700b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 -- 172.21.15.161:0/2528201712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7760009b30 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f776e59c640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777006c8f0 0x7f7770146fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f77600af040 tx=0x7f7760019620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f775effd640 1 -- 172.21.15.161:0/2528201712 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600b8670 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f775effd640 1 -- 172.21.15.161:0/2528201712 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77600b87f0 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f775effd640 1 -- 172.21.15.161:0/2528201712 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77600b8ba0 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77700b35c0 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77700b18e0 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f775effd640 1 -- 172.21.15.161:0/2528201712 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f77600c1400 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f775effd640 1 -- 172.21.15.161:0/2528201712 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f77600b6aa0 con 0x7f777006c8f0 2023-12-16T21:42:18.572 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.566+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777006bb50 con 0x7f777006c8f0 2023-12-16T21:42:18.573 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.570+0000 7f775effd640 1 -- 172.21.15.161:0/2528201712 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f77600dfcf0 con 0x7f777006c8f0 2023-12-16T21:42:18.634 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:18.630+0000 7f623559d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-16T21:42:18.640 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.638+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bccb177c-3eb8-44e6-9694-0b3c11292cff", "id": 7} v 0) v1 -- 0x7f7770068960 con 0x7f777006c8f0 2023-12-16T21:42:18.646 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.642+0000 7f775effd640 1 -- 172.21.15.161:0/2528201712 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bccb177c-3eb8-44e6-9694-0b3c11292cff", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f77600ce030 con 0x7f777006c8f0 2023-12-16T21:42:18.646 INFO:teuthology.orchestra.run.smithi161.stdout:7 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.642+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777006c8f0 msgr2=0x7f7770146fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.642+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777006c8f0 0x7f7770146fd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f77600af040 tx=0x7f7760019620 comp rx=0 tx=0).stop 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.646+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 shutdown_connections 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.646+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77700ba920 0x7f77700b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.646+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f77700b52e0 0x7f77700b2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.646+0000 7f7774cfe640 1 --2- 172.21.15.161:0/2528201712 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f777006c8f0 0x7f7770146fd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.646+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 >> 172.21.15.161:0/2528201712 conn(0x7f777006ab80 msgr2=0x7f77700b90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.646+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 shutdown_connections 2023-12-16T21:42:18.648 INFO:teuthology.orchestra.run.smithi161.stderr:2023-12-16T21:42:18.646+0000 7f7774cfe640 1 -- 172.21.15.161:0/2528201712 wait complete. 2023-12-16T21:42:18.664 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-16T21:42:18.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-16T21:42:18.667 INFO:tasks.ceph.osd.0:Started 2023-12-16T21:42:18.667 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-16T21:42:18.667 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-16T21:42:18.669 INFO:tasks.ceph.osd.1:Started 2023-12-16T21:42:18.669 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-16T21:42:18.669 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-16T21:42:18.672 INFO:tasks.ceph.osd.2:Started 2023-12-16T21:42:18.672 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-16T21:42:18.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-16T21:42:18.674 INFO:tasks.ceph.osd.3:Started 2023-12-16T21:42:18.675 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-16T21:42:18.675 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-16T21:42:18.713 INFO:tasks.ceph.osd.4:Started 2023-12-16T21:42:18.713 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-16T21:42:18.713 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-16T21:42:18.715 INFO:tasks.ceph.osd.5:Started 2023-12-16T21:42:18.715 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-16T21:42:18.716 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-16T21:42:18.718 INFO:tasks.ceph.osd.6:Started 2023-12-16T21:42:18.718 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-16T21:42:18.718 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-16T21:42:18.720 INFO:tasks.ceph.osd.7:Started 2023-12-16T21:42:18.720 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64afe9a640 1 Processor -- start 2023-12-16T21:42:18.827 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64afe9a640 1 -- start start 2023-12-16T21:42:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64afe9a640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f64a80b7df0 0x7f64a81526f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64afe9a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a80b2900 con 0x7f64a80b7960 2023-12-16T21:42:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64afe9a640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64a8073a90 con 0x7f64a80736b0 2023-12-16T21:42:18.828 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64afe9a640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64a806bb10 con 0x7f64a80b7df0 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64ad435640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f64a80b7960 0x7f64a814efc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34316/0 (socket says 172.21.15.46:34316) 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64ad435640 1 -- 172.21.15.46:0/3152715759 learned_addr learned my addr 172.21.15.46:0/3152715759 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64ae437640 1 --2- 172.21.15.46:0/3152715759 >> v2:172.21.15.161:3300/0 conn(0x7f64a80b7df0 0x7f64a81526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64acc34640 1 -- 172.21.15.46:0/3152715759 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393747672 0 0) 0x7f64a80b2900 con 0x7f64a80b7960 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.820+0000 7f64acc34640 1 -- 172.21.15.46:0/3152715759 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6494003530 con 0x7f64a80b7960 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64acc34640 1 -- 172.21.15.46:0/3152715759 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 698258228 0 0) 0x7f6494003530 con 0x7f64a80b7960 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64ae437640 1 -- 172.21.15.46:0/3152715759 >> v1:172.21.15.161:6789/0 conn(0x7f64a80736b0 legacy=0x7f64a81470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64ae437640 1 -- 172.21.15.46:0/3152715759 >> v1:172.21.15.46:6789/0 conn(0x7f64a80b7960 legacy=0x7f64a814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64ae437640 1 -- 172.21.15.46:0/3152715759 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a80af150 con 0x7f64a80b7df0 2023-12-16T21:42:18.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64ae437640 1 --2- 172.21.15.46:0/3152715759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a81526f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009d90 tx=0x7f64a001c750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a54be2859d00aa5 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64acc34640 1 -- 172.21.15.46:0/3152715759 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4001680 con 0x7f64a80b7df0 2023-12-16T21:42:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64acc34640 1 -- 172.21.15.46:0/3152715759 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a00b0400 con 0x7f64a80b7df0 2023-12-16T21:42:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64acc34640 1 -- 172.21.15.46:0/3152715759 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a00b0790 con 0x7f64a80b7df0 2023-12-16T21:42:18.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64afe9a640 1 -- 172.21.15.46:0/3152715759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 msgr2=0x7f64a81526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64afe9a640 1 --2- 172.21.15.46:0/3152715759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a81526f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009d90 tx=0x7f64a001c750 comp rx=0 tx=0).stop 2023-12-16T21:42:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64afe9a640 1 -- 172.21.15.46:0/3152715759 shutdown_connections 2023-12-16T21:42:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64afe9a640 1 --2- 172.21.15.46:0/3152715759 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a81526f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64afe9a640 1 -- 172.21.15.46:0/3152715759 >> 172.21.15.46:0/3152715759 conn(0x7f64a806a580 msgr2=0x7f64a806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64afe9a640 1 -- 172.21.15.46:0/3152715759 shutdown_connections 2023-12-16T21:42:18.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.824+0000 7f64afe9a640 1 -- 172.21.15.46:0/3152715759 wait complete. 2023-12-16T21:42:18.833 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:18.830+0000 7f623559d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 Processor -- start 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 -- start start 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64a80736b0 0x7f64a8071a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a80b7960 0x7f64a806f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a8137e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8137740 con 0x7f64a80b7960 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64a8071260 con 0x7f64a80b7df0 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64a81364d0 con 0x7f64a80736b0 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a80b7960 0x7f64a806f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a80b7960 0x7f64a806f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34142/0 (socket says 172.21.15.46:34142) 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 -- 172.21.15.46:0/2582791061 learned_addr learned my addr 172.21.15.46:0/2582791061 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ae437640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a8137e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64adc36640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64a80736b0 0x7f64a8071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:18.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 -- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64a80736b0 msgr2=0x7f64a8071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64a80736b0 0x7f64a8071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 -- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 msgr2=0x7f64a8137e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a8137e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 -- 172.21.15.46:0/2582791061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a0009a70 con 0x7f64a80b7960 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64adc36640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64a80736b0 0x7f64a8071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ae437640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a8137e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64ad435640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a80b7960 0x7f64a806f030 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c760 tx=0x7f64a400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f649e7fc640 1 -- 172.21.15.46:0/2582791061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4005ae0 con 0x7f64a80b7960 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f649e7fc640 1 -- 172.21.15.46:0/2582791061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a4005c40 con 0x7f64a80b7960 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a8071520 con 0x7f64a80b7960 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f649e7fc640 1 -- 172.21.15.46:0/2582791061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4007070 con 0x7f64a80b7960 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a8135730 con 0x7f64a80b7960 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f649e7fc640 1 -- 172.21.15.46:0/2582791061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f64a40074d0 con 0x7f64a80b7960 2023-12-16T21:42:18.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f649e7fc640 1 -- 172.21.15.46:0/2582791061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f64a401bad0 con 0x7f64a80b7960 2023-12-16T21:42:18.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.828+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64a8071740 con 0x7f64a80b7960 2023-12-16T21:42:18.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.836+0000 7f649e7fc640 1 -- 172.21.15.46:0/2582791061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f64a401b400 con 0x7f64a80b7960 2023-12-16T21:42:18.896 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:18.894+0000 7f623559d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-16T21:42:18.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.936+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64a80688f0 con 0x7f64a80b7960 2023-12-16T21:42:18.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.936+0000 7f649e7fc640 1 -- 172.21.15.46:0/2582791061 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f64a402a090 con 0x7f64a80b7960 2023-12-16T21:42:18.944 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:18.944 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":10,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:18.641021+0000","last_up_change":"0.000000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a80b7960 msgr2=0x7f64a806f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a80b7960 0x7f64a806f030 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f64a400c760 tx=0x7f64a400cc20 comp rx=0 tx=0).stop 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 shutdown_connections 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f64a80b7df0 0x7f64a8137e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f64a80b7960 0x7f64a806f030 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 --2- 172.21.15.46:0/2582791061 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f64a80736b0 0x7f64a8071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 >> 172.21.15.46:0/2582791061 conn(0x7f64a806a580 msgr2=0x7f64a8143a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 shutdown_connections 2023-12-16T21:42:18.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:18.940+0000 7f64afe9a640 1 -- 172.21.15.46:0/2582791061 wait complete. 2023-12-16T21:42:18.959 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-16T21:42:18.960 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-16T21:42:19.020 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:19.018+0000 7f623559d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-16T21:42:19.161 INFO:tasks.ceph.mgr.y.smithi046.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-16T21:42:19.161 INFO:tasks.ceph.mgr.y.smithi046.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-16T21:42:19.161 INFO:tasks.ceph.mgr.y.smithi046.stderr: from numpy import show_config as show_numpy_config 2023-12-16T21:42:19.246 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:19.240+0000 7ff405085180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-16T21:42:19.297 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-16T21:42:19.288+0000 7fbf1c811640 -1 Falling back to public interface 2023-12-16T21:42:19.305 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-12-16T21:42:19.302+0000 7fb45309c640 -1 Falling back to public interface 2023-12-16T21:42:19.309 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-16T21:42:19.304+0000 7fa7cad44640 -1 Falling back to public interface 2023-12-16T21:42:19.310 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-16T21:42:19.304+0000 7ff5e7676640 -1 Falling back to public interface 2023-12-16T21:42:19.312 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-12-16T21:42:19.310+0000 7f28ed4a1640 -1 Falling back to public interface 2023-12-16T21:42:19.317 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-16T21:42:19.312+0000 7f6c69722640 -1 Falling back to public interface 2023-12-16T21:42:19.328 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-12-16T21:42:19.326+0000 7fe9e9449640 -1 Falling back to public interface 2023-12-16T21:42:19.364 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-12-16T21:42:19.362+0000 7f7909b3b640 -1 Falling back to public interface 2023-12-16T21:42:19.428 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:19.420+0000 7ff405085180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-16T21:42:19.606 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:19.600+0000 7ff405085180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-16T21:42:19.636 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:19.634+0000 7f623559d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-16T21:42:19.679 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:19.672+0000 7ff405085180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-16T21:42:19.713 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:19.710+0000 7f623559d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-16T21:42:19.765 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:19.819 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:19.812+0000 7ff405085180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-16T21:42:19.880 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.872+0000 7f860ba92640 1 Processor -- start 2023-12-16T21:42:19.880 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860ba92640 1 -- start start 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860ba92640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f86040b7190 0x7f86040b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860ba92640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8604147b00 0x7f86040b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860ba92640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86040b4940 con 0x7f8604147f80 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860ba92640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040b4f00 con 0x7f8604147b00 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860ba92640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86040b2120 con 0x7f86040b7190 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860902d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8604147b00 0x7f86040b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860a02f640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8604147f80 0x7f86040709c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:34318/0 (socket says 172.21.15.46:34318) 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860a02f640 1 -- 172.21.15.46:0/199273708 learned_addr learned my addr 172.21.15.46:0/199273708 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:19.882 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860982e640 1 --2- 172.21.15.46:0/199273708 >> v2:172.21.15.161:3300/0 conn(0x7f86040b7190 0x7f86040b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:19.883 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860902d640 1 -- 172.21.15.46:0/199273708 >> v2:172.21.15.161:3300/0 conn(0x7f86040b7190 msgr2=0x7f86040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:19.883 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860902d640 1 --2- 172.21.15.46:0/199273708 >> v2:172.21.15.161:3300/0 conn(0x7f86040b7190 0x7f86040b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.883 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860902d640 1 -- 172.21.15.46:0/199273708 >> v1:172.21.15.46:6789/0 conn(0x7f8604147f80 legacy=0x7f86040709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:19.883 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860902d640 1 -- 172.21.15.46:0/199273708 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86040b6480 con 0x7f8604147b00 2023-12-16T21:42:19.883 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860882c640 1 -- 172.21.15.46:0/199273708 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722900916 0 0) 0x7f86040b4940 con 0x7f8604147f80 2023-12-16T21:42:19.884 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860902d640 1 --2- 172.21.15.46:0/199273708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86040b3a10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86040b78f0 tx=0x7f85f401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5738c30f4265011 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:19.884 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860882c640 1 -- 172.21.15.46:0/199273708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40ac630 con 0x7f8604147b00 2023-12-16T21:42:19.884 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860882c640 1 -- 172.21.15.46:0/199273708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f40ac790 con 0x7f8604147b00 2023-12-16T21:42:19.884 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.876+0000 7f860882c640 1 -- 172.21.15.46:0/199273708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40acb20 con 0x7f8604147b00 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- 172.21.15.46:0/199273708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 msgr2=0x7f86040b3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 --2- 172.21.15.46:0/199273708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86040b3a10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f86040b78f0 tx=0x7f85f401c3d0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- 172.21.15.46:0/199273708 shutdown_connections 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 --2- 172.21.15.46:0/199273708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86040b3a10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 --2- 172.21.15.46:0/199273708 >> v2:172.21.15.161:3300/0 conn(0x7f86040b7190 0x7f86040b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- 172.21.15.46:0/199273708 >> 172.21.15.46:0/199273708 conn(0x7f860406a980 msgr2=0x7f860406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- 172.21.15.46:0/199273708 shutdown_connections 2023-12-16T21:42:19.885 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- 172.21.15.46:0/199273708 wait complete. 2023-12-16T21:42:19.886 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:19.882+0000 7f623559d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 Processor -- start 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- start start 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86040b7190 0x7f86041362c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86041dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8604147f80 0x7f8604142400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8604141730 con 0x7f8604147b00 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86040b0770 con 0x7f86040b7190 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8604145200 con 0x7f8604147f80 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86041dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:19.888 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86041dfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34154/0 (socket says 172.21.15.46:34154) 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 -- 172.21.15.46:0/3875578865 learned_addr learned my addr 172.21.15.46:0/3875578865 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860982e640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86040b7190 0x7f86041362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860a02f640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8604147f80 0x7f8604142400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 -- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8604147f80 msgr2=0x7f8604142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8604147f80 0x7f8604142400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 -- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86040b7190 msgr2=0x7f86041362c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86040b7190 0x7f86041362c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 -- 172.21.15.46:0/3875578865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009420 con 0x7f8604147b00 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860902d640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86041dfee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85f401a000 tx=0x7f85f4019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f85fa7fc640 1 -- 172.21.15.46:0/3875578865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40b5840 con 0x7f8604147b00 2023-12-16T21:42:19.889 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f85fa7fc640 1 -- 172.21.15.46:0/3875578865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f40b59c0 con 0x7f8604147b00 2023-12-16T21:42:19.890 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8604138410 con 0x7f8604147b00 2023-12-16T21:42:19.890 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f85fa7fc640 1 -- 172.21.15.46:0/3875578865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f40b5d70 con 0x7f8604147b00 2023-12-16T21:42:19.890 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86041440f0 con 0x7f8604147b00 2023-12-16T21:42:19.890 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f85fa7fc640 1 -- 172.21.15.46:0/3875578865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f85f40b3040 con 0x7f8604147b00 2023-12-16T21:42:19.890 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.880+0000 7f85fa7fc640 1 -- 172.21.15.46:0/3875578865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f85f40abaf0 con 0x7f8604147b00 2023-12-16T21:42:19.890 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.884+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86040679e0 con 0x7f8604147b00 2023-12-16T21:42:19.891 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.884+0000 7f85fa7fc640 1 -- 172.21.15.46:0/3875578865 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72915 (secure 0 0 0) 0x7f85f40dccf0 con 0x7f8604147b00 2023-12-16T21:42:19.948 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.940+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86041418e0 con 0x7f8604147b00 2023-12-16T21:42:19.948 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f85fa7fc640 1 -- 172.21.15.46:0/3875578865 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f86041418e0 con 0x7f8604147b00 2023-12-16T21:42:19.949 INFO:teuthology.misc.health.smithi046.stdout: 2023-12-16T21:42:19.949 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":10,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:18.641021+0000","last_up_change":"0.000000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 msgr2=0x7f86041dfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86041dfee0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f85f401a000 tx=0x7f85f4019170 comp rx=0 tx=0).stop 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 shutdown_connections 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8604147f80 0x7f8604142400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8604147b00 0x7f86041dfee0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 --2- 172.21.15.46:0/3875578865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f86040b7190 0x7f86041362c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 >> 172.21.15.46:0/3875578865 conn(0x7f860406a980 msgr2=0x7f8604066410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 shutdown_connections 2023-12-16T21:42:19.950 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:19.944+0000 7f860ba92640 1 -- 172.21.15.46:0/3875578865 wait complete. 2023-12-16T21:42:19.961 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:19.958+0000 7f623559d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-16T21:42:19.963 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-16T21:42:20.251 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:20.246+0000 7f623559d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-16T21:42:20.312 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:20.310+0000 7f623559d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-16T21:42:20.471 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:20.464+0000 7ff405085180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-16T21:42:20.550 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:20.544+0000 7ff405085180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-16T21:42:20.731 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:20.724+0000 7ff405085180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-16T21:42:20.808 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:20.800+0000 7ff405085180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-16T21:42:21.114 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:21.108+0000 7ff405085180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-16T21:42:21.176 INFO:tasks.ceph.mgr.y.smithi046.stderr:2023-12-16T21:42:21.172+0000 7ff405085180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-16T21:42:21.368 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:21.366+0000 7f61dcd29640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-16T21:42:21.368 INFO:tasks.ceph.mgr.x.smithi161.stderr:2023-12-16T21:42:21.366+0000 7f61dcd29640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-16T21:42:23.152 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-12-16T21:42:23.150+0000 7fb45309c640 -1 osd.4 0 log_to_monitors true 2023-12-16T21:42:23.153 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-12-16T21:42:23.150+0000 7fe9e9449640 -1 osd.7 0 log_to_monitors true 2023-12-16T21:42:23.153 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-16T21:42:23.148+0000 7fbf1c811640 -1 osd.0 0 log_to_monitors true 2023-12-16T21:42:23.156 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-12-16T21:42:23.154+0000 7f28ed4a1640 -1 osd.5 0 log_to_monitors true 2023-12-16T21:42:23.263 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-16T21:42:23.256+0000 7fa7cad44640 -1 osd.2 0 log_to_monitors true 2023-12-16T21:42:23.264 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-16T21:42:23.260+0000 7f6c69722640 -1 osd.1 0 log_to_monitors true 2023-12-16T21:42:23.281 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-12-16T21:42:23.278+0000 7f7909b3b640 -1 osd.6 0 log_to_monitors true 2023-12-16T21:42:23.294 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-16T21:42:23.288+0000 7ff5e7676640 -1 osd.3 0 log_to_monitors true 2023-12-16T21:42:26.186 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-16T21:42:26.180+0000 7fa7c64ca640 -1 osd.2 0 waiting for initial osdmap 2023-12-16T21:42:26.213 INFO:tasks.ceph.osd.2.smithi046.stderr:2023-12-16T21:42:26.208+0000 7fa7beb17640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:26.392 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-16T21:42:26.384+0000 7f6c64ea8640 -1 osd.1 0 waiting for initial osdmap 2023-12-16T21:42:26.417 INFO:tasks.ceph.osd.1.smithi046.stderr:2023-12-16T21:42:26.412+0000 7f6c5d4f5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:26.428 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-16T21:42:26.420+0000 7fbf17f97640 -1 osd.0 0 waiting for initial osdmap 2023-12-16T21:42:26.457 INFO:tasks.ceph.osd.0.smithi046.stderr:2023-12-16T21:42:26.452+0000 7fbf105e4640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:26.485 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-16T21:42:26.480+0000 7ff5e2dfc640 -1 osd.3 0 waiting for initial osdmap 2023-12-16T21:42:26.513 INFO:tasks.ceph.osd.3.smithi046.stderr:2023-12-16T21:42:26.508+0000 7ff5db449640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:26.770 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:26.892 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.884+0000 7fb56e4c1640 1 Processor -- start 2023-12-16T21:42:26.892 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.884+0000 7fb56e4c1640 1 -- start start 2023-12-16T21:42:26.893 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb56e4c1640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb56806c410 0x7fb5681433f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:26.893 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb56e4c1640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5680af510 0x7fb568141f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:26.893 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb56e4c1640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb568142f60 con 0x7fb568143930 2023-12-16T21:42:26.893 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb56e4c1640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56806bbb0 con 0x7fb5680af510 2023-12-16T21:42:26.894 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb56e4c1640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5680afa60 con 0x7fb56806c410 2023-12-16T21:42:26.894 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb56806c410 0x7fb5681433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:26.894 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb56806c410 0x7fb5681433f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:58816/0 (socket says 172.21.15.46:58816) 2023-12-16T21:42:26.894 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb5677fe640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb5680af510 0x7fb568141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:26.894 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb56ca5e640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fb568143930 0x7fb568142490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:32848/0 (socket says 172.21.15.46:32848) 2023-12-16T21:42:26.894 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 -- 172.21.15.46:0/2644325797 learned_addr learned my addr 172.21.15.46:0/2644325797 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:26.894 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-12-16T21:42:26.889+0000 7fe9e4bcf640 -1 osd.7 0 waiting for initial osdmap 2023-12-16T21:42:26.895 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 -- 172.21.15.46:0/2644325797 >> v2:172.21.15.161:3300/0 conn(0x7fb5680af510 msgr2=0x7fb568141f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:26.895 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 --2- 172.21.15.46:0/2644325797 >> v2:172.21.15.161:3300/0 conn(0x7fb5680af510 0x7fb568141f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:26.895 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 -- 172.21.15.46:0/2644325797 >> v1:172.21.15.161:6789/0 conn(0x7fb568143930 legacy=0x7fb568142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:26.895 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 -- 172.21.15.46:0/2644325797 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5680afd40 con 0x7fb56806c410 2023-12-16T21:42:26.895 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb5677fe640 1 --2- 172.21.15.46:0/2644325797 >> v2:172.21.15.161:3300/0 conn(0x7fb5680af510 0x7fb568141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:26.896 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.888+0000 7fb567fff640 1 --2- 172.21.15.46:0/2644325797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb56806c410 0x7fb5681433f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb550009870 tx=0x7fb55001c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d309c336ffd39a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:26.900 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb566ffd640 1 -- 172.21.15.46:0/2644325797 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5500ab600 con 0x7fb56806c410 2023-12-16T21:42:26.901 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb566ffd640 1 -- 172.21.15.46:0/2644325797 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5500a7050 con 0x7fb56806c410 2023-12-16T21:42:26.901 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb566ffd640 1 -- 172.21.15.46:0/2644325797 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5500ab980 con 0x7fb56806c410 2023-12-16T21:42:26.901 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- 172.21.15.46:0/2644325797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb56806c410 msgr2=0x7fb5681433f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:26.901 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/2644325797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb56806c410 0x7fb5681433f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb550009870 tx=0x7fb55001c3f0 comp rx=0 tx=0).stop 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- 172.21.15.46:0/2644325797 shutdown_connections 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/2644325797 >> v2:172.21.15.161:3300/0 conn(0x7fb5680af510 0x7fb568141f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/2644325797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb56806c410 0x7fb5681433f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- 172.21.15.46:0/2644325797 >> 172.21.15.46:0/2644325797 conn(0x7fb5680b5c90 msgr2=0x7fb5680b60a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- 172.21.15.46:0/2644325797 shutdown_connections 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- 172.21.15.46:0/2644325797 wait complete. 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 Processor -- start 2023-12-16T21:42:26.902 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- start start 2023-12-16T21:42:26.904 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb56806c410 0x7fb568140b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:26.904 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5680af510 0x7fb5681dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:26.904 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb568143930 0x7fb5681dd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:26.904 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56813cc60 con 0x7fb568143930 2023-12-16T21:42:26.904 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56806eb20 con 0x7fb56806c410 2023-12-16T21:42:26.904 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56e4c1640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5680718e0 con 0x7fb5680af510 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb568143930 0x7fb5681dd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb568143930 0x7fb5681dd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34186/0 (socket says 172.21.15.46:34186) 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 -- 172.21.15.46:0/1190465387 learned_addr learned my addr 172.21.15.46:0/1190465387 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb567fff640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb56806c410 0x7fb568140b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb5677fe640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5680af510 0x7fb5681dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 -- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5680af510 msgr2=0x7fb5681dcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5680af510 0x7fb5681dcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 -- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb56806c410 msgr2=0x7fb568140b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb56806c410 0x7fb568140b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 -- 172.21.15.46:0/1190465387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb558009a70 con 0x7fb568143930 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb56ca5e640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb568143930 0x7fb5681dd010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c420 tx=0x7fb56000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.896+0000 7fb564ff9640 1 -- 172.21.15.46:0/1190465387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5600074e0 con 0x7fb568143930 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb564ff9640 1 -- 172.21.15.46:0/1190465387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb560007640 con 0x7fb568143930 2023-12-16T21:42:26.905 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5500094d0 con 0x7fb568143930 2023-12-16T21:42:26.906 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb564ff9640 1 -- 172.21.15.46:0/1190465387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb560004a10 con 0x7fb568143930 2023-12-16T21:42:26.906 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56806f8f0 con 0x7fb568143930 2023-12-16T21:42:26.906 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5680679e0 con 0x7fb568143930 2023-12-16T21:42:26.907 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb564ff9640 1 -- 172.21.15.46:0/1190465387 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) v1 ==== 85798+0+0 (secure 0 0 0) 0x7fb560004b70 con 0x7fb568143930 2023-12-16T21:42:26.907 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb564ff9640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb5480663e0 0x7fb5480688a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:26.907 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb564ff9640 1 -- 172.21.15.46:0/1190465387 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fb5600827f0 con 0x7fb568143930 2023-12-16T21:42:26.907 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb567fff640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb5480663e0 0x7fb5480688a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:26.907 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.900+0000 7fb567fff640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb5480663e0 0x7fb5480688a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb5500ad000 tx=0x7fb5500a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:26.910 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:26.904+0000 7fb564ff9640 1 -- 172.21.15.46:0/1190465387 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fb560050b80 con 0x7fb568143930 2023-12-16T21:42:26.919 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-12-16T21:42:26.913+0000 7f79052c1640 -1 osd.6 0 waiting for initial osdmap 2023-12-16T21:42:26.933 INFO:tasks.ceph.osd.7.smithi161.stderr:2023-12-16T21:42:26.929+0000 7fe9dd21c640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:26.940 INFO:tasks.ceph.osd.6.smithi161.stderr:2023-12-16T21:42:26.937+0000 7f78fd90e640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:26.948 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-12-16T21:42:26.945+0000 7fb450058640 -1 osd.4 0 waiting for initial osdmap 2023-12-16T21:42:26.976 INFO:tasks.ceph.osd.4.smithi161.stderr:2023-12-16T21:42:26.973+0000 7fb446e6f640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:27.011 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-12-16T21:42:27.005+0000 7f28ea45d640 -1 osd.5 0 waiting for initial osdmap 2023-12-16T21:42:27.041 INFO:tasks.ceph.osd.5.smithi161.stderr:2023-12-16T21:42:27.037+0000 7f28e1274640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-16T21:42:27.062 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.056+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb56813cda0 con 0x7fb568143930 2023-12-16T21:42:27.063 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.056+0000 7fb564ff9640 1 -- 172.21.15.46:0/1190465387 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb5600541a0 con 0x7fb568143930 2023-12-16T21:42:27.064 INFO:teuthology.misc.health.smithi046.stdout: 2023-12-16T21:42:27.064 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":12,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:24.663638+0000","last_up_change":"0.000000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb5480663e0 msgr2=0x7fb5480688a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb5480663e0 0x7fb5480688a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb5500ad000 tx=0x7fb5500a6000 comp rx=0 tx=0).stop 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb568143930 msgr2=0x7fb5681dd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb568143930 0x7fb5681dd010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c420 tx=0x7fb56000c8e0 comp rx=0 tx=0).stop 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 shutdown_connections 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb5480663e0 0x7fb5480688a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb568143930 0x7fb5681dd010 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb5680af510 0x7fb5681dcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 --2- 172.21.15.46:0/1190465387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb56806c410 0x7fb568140b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 >> 172.21.15.46:0/1190465387 conn(0x7fb5680b5c90 msgr2=0x7fb568142170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:27.066 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 shutdown_connections 2023-12-16T21:42:27.067 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:27.060+0000 7fb56e4c1640 1 -- 172.21.15.46:0/1190465387 wait complete. 2023-12-16T21:42:27.080 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-16T21:42:33.887 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:34.013 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 Processor -- start 2023-12-16T21:42:34.013 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 -- start start 2023-12-16T21:42:34.014 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72a00b7190 0x7f72a0147b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.014 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f72a00b5ba0 0x7f72a00b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.014 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a00708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.014 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a00b1770 con 0x7f72a00b7190 2023-12-16T21:42:34.014 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a00b5040 con 0x7f72a00b5ba0 2023-12-16T21:42:34.014 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a62e2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0148040 con 0x7f72a00b41c0 2023-12-16T21:42:34.015 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72a00b7190 0x7f72a0147b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.015 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f72a00b7190 0x7f72a0147b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44726/0 (socket says 172.21.15.46:44726) 2023-12-16T21:42:34.015 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 -- 172.21.15.46:0/1370676149 learned_addr learned my addr 172.21.15.46:0/1370676149 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:34.015 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f72a487f640 1 --2- 172.21.15.46:0/1370676149 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a00708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.015 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729f7fe640 1 --2- 172.21.15.46:0/1370676149 >> v2:172.21.15.161:3300/0 conn(0x7f72a00b5ba0 0x7f72a00b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.015 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 -- 172.21.15.46:0/1370676149 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 msgr2=0x7f72a00708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:34.015 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 --2- 172.21.15.46:0/1370676149 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a00708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.016 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 -- 172.21.15.46:0/1370676149 >> v2:172.21.15.161:3300/0 conn(0x7f72a00b5ba0 msgr2=0x7f72a00b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:34.016 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 --2- 172.21.15.46:0/1370676149 >> v2:172.21.15.161:3300/0 conn(0x7f72a00b5ba0 0x7f72a00b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.016 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.008+0000 7f729ffff640 1 -- 172.21.15.46:0/1370676149 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a00724c0 con 0x7f72a00b7190 2023-12-16T21:42:34.019 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f729ffff640 1 --2- 172.21.15.46:0/1370676149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a00b7190 0x7f72a0147b00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f72a0071ef0 tx=0x7f729001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a071b3649280e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.019 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f729effd640 1 -- 172.21.15.46:0/1370676149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900a7070 con 0x7f72a00b7190 2023-12-16T21:42:34.019 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f729effd640 1 -- 172.21.15.46:0/1370676149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72900ab430 con 0x7f72a00b7190 2023-12-16T21:42:34.019 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f729effd640 1 -- 172.21.15.46:0/1370676149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72900ab7c0 con 0x7f72a00b7190 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 -- 172.21.15.46:0/1370676149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a00b7190 msgr2=0x7f72a0147b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 --2- 172.21.15.46:0/1370676149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a00b7190 0x7f72a0147b00 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f72a0071ef0 tx=0x7f729001c470 comp rx=0 tx=0).stop 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 -- 172.21.15.46:0/1370676149 shutdown_connections 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 --2- 172.21.15.46:0/1370676149 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a00708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 --2- 172.21.15.46:0/1370676149 >> v2:172.21.15.161:3300/0 conn(0x7f72a00b5ba0 0x7f72a00b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 --2- 172.21.15.46:0/1370676149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a00b7190 0x7f72a0147b00 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 -- 172.21.15.46:0/1370676149 >> 172.21.15.46:0/1370676149 conn(0x7f72a006a980 msgr2=0x7f72a006ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:34.020 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 -- 172.21.15.46:0/1370676149 shutdown_connections 2023-12-16T21:42:34.021 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.012+0000 7f72a62e2640 1 -- 172.21.15.46:0/1370676149 wait complete. 2023-12-16T21:42:34.021 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 Processor -- start 2023-12-16T21:42:34.021 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 -- start start 2023-12-16T21:42:34.023 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a01379a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a00b7190 0x7f72a0138d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a01420e0 0x7f72a013b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a0131db0 con 0x7f72a01420e0 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a013a0b0 con 0x7f72a00b7190 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a01dd4d0 con 0x7f72a00b41c0 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a01420e0 0x7f72a013b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a01420e0 0x7f72a013b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44730/0 (socket says 172.21.15.46:44730) 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 -- 172.21.15.46:0/3772513734 learned_addr learned my addr 172.21.15.46:0/3772513734 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729ffff640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a01379a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729f7fe640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a00b7190 0x7f72a0138d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 -- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 msgr2=0x7f72a01379a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a01379a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 -- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a00b7190 msgr2=0x7f72a0138d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.024 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a00b7190 0x7f72a0138d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 -- 172.21.15.46:0/3772513734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72900094d0 con 0x7f72a01420e0 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729ffff640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a01379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729f7fe640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a00b7190 0x7f72a0138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a487f640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a01420e0 0x7f72a013b750 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f729800efc0 tx=0x7f729800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729cff9640 1 -- 172.21.15.46:0/3772513734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009ca0 con 0x7f72a01420e0 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729cff9640 1 -- 172.21.15.46:0/3772513734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729800f040 con 0x7f72a01420e0 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729cff9640 1 -- 172.21.15.46:0/3772513734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980146c0 con 0x7f72a01420e0 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a0133ed0 con 0x7f72a01420e0 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a0137ee0 con 0x7f72a01420e0 2023-12-16T21:42:34.025 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a0068900 con 0x7f72a01420e0 2023-12-16T21:42:34.026 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.016+0000 7f729cff9640 1 -- 172.21.15.46:0/3772513734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7298014820 con 0x7f72a01420e0 2023-12-16T21:42:34.026 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.020+0000 7f729cff9640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f728c066470 0x7f728c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.026 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.020+0000 7f729cff9640 1 -- 172.21.15.46:0/3772513734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f72980849e0 con 0x7f72a01420e0 2023-12-16T21:42:34.026 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.020+0000 7f729ffff640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f728c066470 0x7f728c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.026 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.020+0000 7f729ffff640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f728c066470 0x7f728c068930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72900af000 tx=0x7f7290008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.029 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.024+0000 7f729cff9640 1 -- 172.21.15.46:0/3772513734 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7298052240 con 0x7f72a01420e0 2023-12-16T21:42:34.184 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.176+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72a01438c0 con 0x7f72a01420e0 2023-12-16T21:42:34.185 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f729cff9640 1 -- 172.21.15.46:0/3772513734 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f7298055860 con 0x7f72a01420e0 2023-12-16T21:42:34.186 INFO:teuthology.misc.health.smithi046.stdout: 2023-12-16T21:42:34.186 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":17,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:31.313784+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f728c066470 msgr2=0x7f728c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f728c066470 0x7f728c068930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72900af000 tx=0x7f7290008040 comp rx=0 tx=0).stop 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a01420e0 msgr2=0x7f72a013b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a01420e0 0x7f72a013b750 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f729800efc0 tx=0x7f729800c490 comp rx=0 tx=0).stop 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 shutdown_connections 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f728c066470 0x7f728c068930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f72a01420e0 0x7f72a013b750 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.188 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f72a00b7190 0x7f72a0138d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.189 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 --2- 172.21.15.46:0/3772513734 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f72a00b41c0 0x7f72a01379a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.189 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 >> 172.21.15.46:0/3772513734 conn(0x7f72a006a980 msgr2=0x7f72a00ba600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:34.189 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 shutdown_connections 2023-12-16T21:42:34.189 INFO:teuthology.misc.health.smithi046.stderr:2023-12-16T21:42:34.180+0000 7f72a62e2640 1 -- 172.21.15.46:0/3772513734 wait complete. 2023-12-16T21:42:34.204 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-16T21:42:34.204 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-16T21:42:34.205 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-16T21:42:34.205 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-16T21:42:34.207 INFO:tasks.ceph.mds.a:Started 2023-12-16T21:42:34.207 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-16T21:42:34.207 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-16T21:42:34.210 INFO:tasks.ceph.mds.b:Started 2023-12-16T21:42:34.210 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-16T21:42:34.210 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:34.300 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-16T21:42:34.310 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-12-16T21:42:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd8bca640 1 Processor -- start 2023-12-16T21:42:34.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd8bca640 1 -- start start 2023-12-16T21:42:34.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd8bca640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b5d50 0x7f8cd41406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd8bca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cd4140c10 0x7f8cd4071970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd8bca640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd40b6130 con 0x7f8cd406ffe0 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd8bca640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd40b4560 con 0x7f8cd4140c10 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd8bca640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40b4840 con 0x7f8cd40b5d50 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cd4140c10 0x7f8cd4071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd2d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8cd406ffe0 0x7f8cd406a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59228/0 (socket says 172.21.15.46:59228) 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8cd4140c10 0x7f8cd4071970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44756/0 (socket says 172.21.15.46:44756) 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd2d9d640 1 -- 172.21.15.46:0/967738984 learned_addr learned my addr 172.21.15.46:0/967738984 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd259c640 1 --2- 172.21.15.46:0/967738984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b5d50 0x7f8cd41406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd1d9b640 1 -- 172.21.15.46:0/967738984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b5d50 msgr2=0x7f8cd41406d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd1d9b640 1 --2- 172.21.15.46:0/967738984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b5d50 0x7f8cd41406d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd1d9b640 1 -- 172.21.15.46:0/967738984 >> v1:172.21.15.46:6789/0 conn(0x7f8cd406ffe0 legacy=0x7f8cd406a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd1d9b640 1 -- 172.21.15.46:0/967738984 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40b4b20 con 0x7f8cd4140c10 2023-12-16T21:42:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd1d9b640 1 --2- 172.21.15.46:0/967738984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4140c10 0x7f8cd4071970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80097c0 tx=0x7f8cc801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e332e6a4b8d4119 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.320+0000 7f8cd159a640 1 -- 172.21.15.46:0/967738984 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80ab600 con 0x7f8cd4140c10 2023-12-16T21:42:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd159a640 1 -- 172.21.15.46:0/967738984 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc80a7050 con 0x7f8cd4140c10 2023-12-16T21:42:34.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd159a640 1 -- 172.21.15.46:0/967738984 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80ab980 con 0x7f8cd4140c10 2023-12-16T21:42:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- 172.21.15.46:0/967738984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4140c10 msgr2=0x7f8cd4071970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/967738984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4140c10 0x7f8cd4071970 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80097c0 tx=0x7f8cc801c280 comp rx=0 tx=0).stop 2023-12-16T21:42:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- 172.21.15.46:0/967738984 shutdown_connections 2023-12-16T21:42:34.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/967738984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd4140c10 0x7f8cd4071970 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/967738984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b5d50 0x7f8cd41406d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- 172.21.15.46:0/967738984 >> 172.21.15.46:0/967738984 conn(0x7f8cd40b91c0 msgr2=0x7f8cd40b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- 172.21.15.46:0/967738984 shutdown_connections 2023-12-16T21:42:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- 172.21.15.46:0/967738984 wait complete. 2023-12-16T21:42:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 Processor -- start 2023-12-16T21:42:34.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- start start 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd406ffe0 0x7f8cd4139a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd40b5d50 0x7f8cd40b8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b1920 0x7f8cd40b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd4133c90 con 0x7f8cd406ffe0 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd413a5e0 con 0x7f8cd40b5d50 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40b2410 con 0x7f8cd40b1920 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd406ffe0 0x7f8cd4139a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd406ffe0 0x7f8cd4139a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44772/0 (socket says 172.21.15.46:44772) 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 -- 172.21.15.46:0/488637568 learned_addr learned my addr 172.21.15.46:0/488637568 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd1d9b640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd40b5d50 0x7f8cd40b8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd2d9d640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b1920 0x7f8cd40b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 -- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b1920 msgr2=0x7f8cd40b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b1920 0x7f8cd40b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 -- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd40b5d50 msgr2=0x7f8cd40b8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd40b5d50 0x7f8cd40b8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 -- 172.21.15.46:0/488637568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc8009420 con 0x7f8cd406ffe0 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd2d9d640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b1920 0x7f8cd40b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd259c640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd406ffe0 0x7f8cd4139a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e930 tx=0x7f8cc400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cc2ffd640 1 -- 172.21.15.46:0/488637568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4009690 con 0x7f8cd406ffe0 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cc2ffd640 1 -- 172.21.15.46:0/488637568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc400ca00 con 0x7f8cd406ffe0 2023-12-16T21:42:34.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cc2ffd640 1 -- 172.21.15.46:0/488637568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc400cd70 con 0x7f8cd406ffe0 2023-12-16T21:42:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd40b26f0 con 0x7f8cd406ffe0 2023-12-16T21:42:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.324+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd40b0840 con 0x7f8cd406ffe0 2023-12-16T21:42:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.328+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd40679e0 con 0x7f8cd406ffe0 2023-12-16T21:42:34.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.328+0000 7f8cc2ffd640 1 -- 172.21.15.46:0/488637568 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8cc4013470 con 0x7f8cd406ffe0 2023-12-16T21:42:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.328+0000 7f8cc2ffd640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8cb4066470 0x7f8cb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.328+0000 7f8cc2ffd640 1 -- 172.21.15.46:0/488637568 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8cc4083c40 con 0x7f8cd406ffe0 2023-12-16T21:42:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.328+0000 7f8cd1d9b640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8cb4066470 0x7f8cb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.328+0000 7f8cd1d9b640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8cb4066470 0x7f8cb4068930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ad000 tx=0x7f8cc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.332+0000 7f8cc2ffd640 1 -- 172.21.15.46:0/488637568 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8cc4051610 con 0x7f8cd406ffe0 2023-12-16T21:42:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cd4067e80 con 0x7f8cd406ffe0 2023-12-16T21:42:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cc2ffd640 1 -- 172.21.15.46:0/488637568 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f8cc4017090 con 0x7f8cd406ffe0 2023-12-16T21:42:34.498 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:34.498 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:31.313784+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8cb4066470 msgr2=0x7f8cb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8cb4066470 0x7f8cb4068930 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cc80ad000 tx=0x7f8cc8008040 comp rx=0 tx=0).stop 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd406ffe0 msgr2=0x7f8cd4139a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd406ffe0 0x7f8cd4139a50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400e930 tx=0x7f8cc400edf0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 shutdown_connections 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8cb4066470 0x7f8cb4068930 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8cd40b1920 0x7f8cd40b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8cd40b5d50 0x7f8cd40b8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 --2- 172.21.15.46:0/488637568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8cd406ffe0 0x7f8cd4139a50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 >> 172.21.15.46:0/488637568 conn(0x7f8cd40b91c0 msgr2=0x7f8cd4072030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 shutdown_connections 2023-12-16T21:42:34.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.492+0000 7f8cd8bca640 1 -- 172.21.15.46:0/488637568 wait complete. 2023-12-16T21:42:34.517 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:34.517 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.628+0000 7f213c552640 1 Processor -- start 2023-12-16T21:42:34.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.628+0000 7f213c552640 1 -- start start 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213c552640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f21340bc8f0 0x7f21340b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213c552640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f213414a0a0 0x7f21341406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213c552640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213414a4b0 con 0x7f2134149bf0 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213c552640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21340b3620 con 0x7f213414a0a0 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213c552640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21340bccd0 con 0x7f21340bc8f0 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f213414a0a0 0x7f21341406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f213414a0a0 0x7f21341406c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44776/0 (socket says 172.21.15.46:44776) 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 -- 172.21.15.46:0/867617660 learned_addr learned my addr 172.21.15.46:0/867617660 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213a2ee640 1 --2- 172.21.15.46:0/867617660 >> v2:172.21.15.161:3300/0 conn(0x7f21340bc8f0 0x7f21340b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 -- 172.21.15.46:0/867617660 >> v2:172.21.15.161:3300/0 conn(0x7f21340bc8f0 msgr2=0x7f21340b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 --2- 172.21.15.46:0/867617660 >> v2:172.21.15.161:3300/0 conn(0x7f21340bc8f0 0x7f21340b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 -- 172.21.15.46:0/867617660 >> v1:172.21.15.161:6789/0 conn(0x7f2134149bf0 legacy=0x7f213413ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 -- 172.21.15.46:0/867617660 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21340b60d0 con 0x7f213414a0a0 2023-12-16T21:42:34.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213a2ee640 1 --2- 172.21.15.46:0/867617660 >> v2:172.21.15.161:3300/0 conn(0x7f21340bc8f0 0x7f21340b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f213aaef640 1 --2- 172.21.15.46:0/867617660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341406c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f21340b83a0 tx=0x7f212c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194959e85836524c server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f21392ec640 1 -- 172.21.15.46:0/867617660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0a5740 con 0x7f213414a0a0 2023-12-16T21:42:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f21392ec640 1 -- 172.21.15.46:0/867617660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c0a58a0 con 0x7f213414a0a0 2023-12-16T21:42:34.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.632+0000 7f21392ec640 1 -- 172.21.15.46:0/867617660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0a5c30 con 0x7f213414a0a0 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- 172.21.15.46:0/867617660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 msgr2=0x7f21341406c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 --2- 172.21.15.46:0/867617660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341406c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f21340b83a0 tx=0x7f212c01c990 comp rx=0 tx=0).stop 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- 172.21.15.46:0/867617660 shutdown_connections 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 --2- 172.21.15.46:0/867617660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341406c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 --2- 172.21.15.46:0/867617660 >> v2:172.21.15.161:3300/0 conn(0x7f21340bc8f0 0x7f21340b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- 172.21.15.46:0/867617660 >> 172.21.15.46:0/867617660 conn(0x7f21340722b0 msgr2=0x7f21340bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- 172.21.15.46:0/867617660 shutdown_connections 2023-12-16T21:42:34.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- 172.21.15.46:0/867617660 wait complete. 2023-12-16T21:42:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 Processor -- start 2023-12-16T21:42:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- start start 2023-12-16T21:42:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21340bc8f0 0x7f213413ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2134149bf0 0x7f21341e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21340b23b0 con 0x7f213414a0a0 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213413d770 con 0x7f2134149bf0 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21341e6340 con 0x7f21340bc8f0 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341e2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44790/0 (socket says 172.21.15.46:44790) 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 -- 172.21.15.46:0/2131954904 learned_addr learned my addr 172.21.15.46:0/2131954904 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 -- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21340bc8f0 msgr2=0x7f213413ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213a2ee640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21340bc8f0 0x7f213413ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f2139aed640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2134149bf0 0x7f21341e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21340bc8f0 0x7f213413ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 -- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2134149bf0 msgr2=0x7f21341e27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2134149bf0 0x7f21341e27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 -- 172.21.15.46:0/2131954904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c009b30 con 0x7f213414a0a0 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213aaef640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341e2ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f212c0af040 tx=0x7f212c019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f212affd640 1 -- 172.21.15.46:0/2131954904 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0b8670 con 0x7f213414a0a0 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f212affd640 1 -- 172.21.15.46:0/2131954904 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212c0b87d0 con 0x7f213414a0a0 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f212affd640 1 -- 172.21.15.46:0/2131954904 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212c0b8b80 con 0x7f213414a0a0 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21341e6620 con 0x7f213414a0a0 2023-12-16T21:42:34.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.636+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213413da50 con 0x7f213414a0a0 2023-12-16T21:42:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.640+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2134068960 con 0x7f213414a0a0 2023-12-16T21:42:34.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.640+0000 7f212affd640 1 -- 172.21.15.46:0/2131954904 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f212c0b8ce0 con 0x7f213414a0a0 2023-12-16T21:42:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.640+0000 7f212affd640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f21180664a0 0x7f2118068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.640+0000 7f212affd640 1 -- 172.21.15.46:0/2131954904 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f212c11f080 con 0x7f213414a0a0 2023-12-16T21:42:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.640+0000 7f213a2ee640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f21180664a0 0x7f2118068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.640+0000 7f213a2ee640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f21180664a0 0x7f2118068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2130006fd0 tx=0x7f2130008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.644+0000 7f212affd640 1 -- 172.21.15.46:0/2131954904 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f212c0eca00 con 0x7f213414a0a0 2023-12-16T21:42:34.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f21340b8620 con 0x7f213414a0a0 2023-12-16T21:42:34.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f212affd640 1 -- 172.21.15.46:0/2131954904 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f212c0f0020 con 0x7f213414a0a0 2023-12-16T21:42:34.798 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f21180664a0 msgr2=0x7f2118068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f21180664a0 0x7f2118068960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2130006fd0 tx=0x7f2130008040 comp rx=0 tx=0).stop 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 msgr2=0x7f21341e2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341e2ce0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f212c0af040 tx=0x7f212c019730 comp rx=0 tx=0).stop 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 shutdown_connections 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f21180664a0 0x7f2118068960 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f213414a0a0 0x7f21341e2ce0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2134149bf0 0x7f21341e27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 --2- 172.21.15.46:0/2131954904 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21340bc8f0 0x7f213413ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 >> 172.21.15.46:0/2131954904 conn(0x7f21340722b0 msgr2=0x7f2134146f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 shutdown_connections 2023-12-16T21:42:34.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.792+0000 7f213c552640 1 -- 172.21.15.46:0/2131954904 wait complete. 2023-12-16T21:42:34.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:42:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 Processor -- start 2023-12-16T21:42:34.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 -- start start 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be00ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f6be00af2e0 0x7f6be00ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6be00b4430 0x7f6be0147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00afc20 con 0x7f6be00b4430 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be00af770 con 0x7f6be00af2e0 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be6f01640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be00aeea0 con 0x7f6be006c920 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6be00b4430 0x7f6be0147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f6be00b4430 0x7f6be0147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44792/0 (socket says 172.21.15.46:44792) 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 -- 172.21.15.46:0/3116302699 learned_addr learned my addr 172.21.15.46:0/3116302699 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be4c9d640 1 --2- 172.21.15.46:0/3116302699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be00ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6bdffff640 1 --2- 172.21.15.46:0/3116302699 >> v2:172.21.15.161:3300/0 conn(0x7f6be00af2e0 0x7f6be00ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 -- 172.21.15.46:0/3116302699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 msgr2=0x7f6be00ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 --2- 172.21.15.46:0/3116302699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be00ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 -- 172.21.15.46:0/3116302699 >> v2:172.21.15.161:3300/0 conn(0x7f6be00af2e0 msgr2=0x7f6be00ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:34.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 --2- 172.21.15.46:0/3116302699 >> v2:172.21.15.161:3300/0 conn(0x7f6be00af2e0 0x7f6be00ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.932+0000 7f6be549e640 1 -- 172.21.15.46:0/3116302699 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be006cd00 con 0x7f6be00b4430 2023-12-16T21:42:34.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be549e640 1 --2- 172.21.15.46:0/3116302699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0147080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6be006cef0 tx=0x7f6bd801ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283b9fcfa4040d0c server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6bdf7fe640 1 -- 172.21.15.46:0/3116302699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80a7b40 con 0x7f6be00b4430 2023-12-16T21:42:34.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6bdf7fe640 1 -- 172.21.15.46:0/3116302699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd80a7ca0 con 0x7f6be00b4430 2023-12-16T21:42:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6bdf7fe640 1 -- 172.21.15.46:0/3116302699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd80af660 con 0x7f6be00b4430 2023-12-16T21:42:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- 172.21.15.46:0/3116302699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 msgr2=0x7f6be0147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3116302699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0147080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6be006cef0 tx=0x7f6bd801ea90 comp rx=0 tx=0).stop 2023-12-16T21:42:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- 172.21.15.46:0/3116302699 shutdown_connections 2023-12-16T21:42:34.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3116302699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0147080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3116302699 >> v2:172.21.15.161:3300/0 conn(0x7f6be00af2e0 0x7f6be00ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3116302699 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- 172.21.15.46:0/3116302699 >> 172.21.15.46:0/3116302699 conn(0x7f6be00b48e0 msgr2=0x7f6be00b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- 172.21.15.46:0/3116302699 shutdown_connections 2023-12-16T21:42:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- 172.21.15.46:0/3116302699 wait complete. 2023-12-16T21:42:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 Processor -- start 2023-12-16T21:42:34.943 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- start start 2023-12-16T21:42:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be009aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0099fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be013ff80 0x7f6be01de170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be009b4c0 con 0x7f6be00b4430 2023-12-16T21:42:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be009b6b0 con 0x7f6be013ff80 2023-12-16T21:42:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be6f01640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be0142880 con 0x7f6be006c920 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6bdffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0099fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6bdffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0099fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44804/0 (socket says 172.21.15.46:44804) 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6bdffff640 1 -- 172.21.15.46:0/3486414173 learned_addr learned my addr 172.21.15.46:0/3486414173 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.936+0000 7f6be4c9d640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be009aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdffff640 1 -- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 msgr2=0x7f6be009aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6be549e640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be013ff80 0x7f6be01de170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdffff640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be009aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdffff640 1 -- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be013ff80 msgr2=0x7f6be01de170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdffff640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be013ff80 0x7f6be01de170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdffff640 1 -- 172.21.15.46:0/3486414173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd800b770 con 0x7f6be00b4430 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6be4c9d640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be009aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdffff640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0099fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0007ef0 tx=0x7f6bd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdd7fa640 1 -- 172.21.15.46:0/3486414173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0014620 con 0x7f6be00b4430 2023-12-16T21:42:34.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdd7fa640 1 -- 172.21.15.46:0/3486414173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd0014780 con 0x7f6be00b4430 2023-12-16T21:42:34.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdd7fa640 1 -- 172.21.15.46:0/3486414173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0014b30 con 0x7f6be00b4430 2023-12-16T21:42:34.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6be0142b60 con 0x7f6be00b4430 2023-12-16T21:42:34.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be009a500 con 0x7f6be00b4430 2023-12-16T21:42:34.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be00679e0 con 0x7f6be00b4430 2023-12-16T21:42:34.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdd7fa640 1 -- 172.21.15.46:0/3486414173 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f6bd0014c90 con 0x7f6be00b4430 2023-12-16T21:42:34.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdd7fa640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6bc0066470 0x7f6bc0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:34.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6bdd7fa640 1 -- 172.21.15.46:0/3486414173 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6bd0084500 con 0x7f6be00b4430 2023-12-16T21:42:34.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.940+0000 7f6be4c9d640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6bc0066470 0x7f6bc0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:34.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.944+0000 7f6be4c9d640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6bc0066470 0x7f6bc0068930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80097c0 tx=0x7f6bc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:34.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:34.944+0000 7f6bdd7fa640 1 -- 172.21.15.46:0/3486414173 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f6bd0051cc0 con 0x7f6be00b4430 2023-12-16T21:42:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.128+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6be00688f0 con 0x7f6be00b4430 2023-12-16T21:42:35.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6bdd7fa640 1 -- 172.21.15.46:0/3486414173 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f6bd00552e0 con 0x7f6be00b4430 2023-12-16T21:42:35.137 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:35.137 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-12-16T21:42:35.137 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 2 2023-12-16T21:42:35.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6bc0066470 msgr2=0x7f6bc0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6bc0066470 0x7f6bc0068930 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6bc80097c0 tx=0x7f6bc8008040 comp rx=0 tx=0).stop 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 msgr2=0x7f6be0099fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0099fc0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0007ef0 tx=0x7f6bd000e4b0 comp rx=0 tx=0).stop 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 shutdown_connections 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6bc0066470 0x7f6bc0068930 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6be013ff80 0x7f6be01de170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6be00b4430 0x7f6be0099fc0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 --2- 172.21.15.46:0/3486414173 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6be006c920 0x7f6be009aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 >> 172.21.15.46:0/3486414173 conn(0x7f6be00b48e0 msgr2=0x7f6be013ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:35.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 shutdown_connections 2023-12-16T21:42:35.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:35.132+0000 7f6be6f01640 1 -- 172.21.15.46:0/3486414173 wait complete. 2023-12-16T21:42:36.549 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:42:36.553 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:42:37.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:42:37.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.280+0000 7f7d8d1c2640 1 Processor -- start 2023-12-16T21:42:37.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.280+0000 7f7d8d1c2640 1 -- start start 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8d1c2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7d880ae5b0 0x7f7d880ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8d1c2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880b3230 0x7f7d880b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8d1c2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880b27e0 con 0x7f7d8806c6c0 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8d1c2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880afe40 con 0x7f7d880ae5b0 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8d1c2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880b3bb0 con 0x7f7d880b3230 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8759e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7d8806c6c0 0x7f7d880b17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59240/0 (socket says 172.21.15.46:59240) 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8759e640 1 -- 172.21.15.46:0/826888266 learned_addr learned my addr 172.21.15.46:0/826888266 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d86d9d640 1 --2- 172.21.15.46:0/826888266 >> v2:172.21.15.46:3300/0 conn(0x7f7d880ae5b0 0x7f7d880ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d8659c640 1 --2- 172.21.15.46:0/826888266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880b3230 0x7f7d880b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d86d9d640 1 -- 172.21.15.46:0/826888266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880b3230 msgr2=0x7f7d880b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d86d9d640 1 --2- 172.21.15.46:0/826888266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880b3230 0x7f7d880b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d86d9d640 1 -- 172.21.15.46:0/826888266 >> v1:172.21.15.46:6789/0 conn(0x7f7d8806c6c0 legacy=0x7f7d880b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d86d9d640 1 -- 172.21.15.46:0/826888266 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880b2a30 con 0x7f7d880ae5b0 2023-12-16T21:42:37.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d85d9b640 1 -- 172.21.15.46:0/826888266 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018192312 0 0) 0x7f7d880b27e0 con 0x7f7d8806c6c0 2023-12-16T21:42:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d86d9d640 1 --2- 172.21.15.46:0/826888266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d880ae5b0 0x7f7d880ae990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7d780097f0 tx=0x7f7d7801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fa35372b8f3541 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d85d9b640 1 -- 172.21.15.46:0/826888266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780ac620 con 0x7f7d880ae5b0 2023-12-16T21:42:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d85d9b640 1 -- 172.21.15.46:0/826888266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d780ac780 con 0x7f7d880ae5b0 2023-12-16T21:42:37.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.284+0000 7f7d85d9b640 1 -- 172.21.15.46:0/826888266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780acb10 con 0x7f7d880ae5b0 2023-12-16T21:42:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/826888266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d880ae5b0 msgr2=0x7f7d880ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/826888266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d880ae5b0 0x7f7d880ae990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7d780097f0 tx=0x7f7d7801c3c0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/826888266 shutdown_connections 2023-12-16T21:42:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/826888266 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880b3230 0x7f7d880b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/826888266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d880ae5b0 0x7f7d880ae990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/826888266 >> 172.21.15.46:0/826888266 conn(0x7f7d880b48e0 msgr2=0x7f7d880b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/826888266 shutdown_connections 2023-12-16T21:42:37.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/826888266 wait complete. 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 Processor -- start 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- start start 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d8806c6c0 0x7f7d881dd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880ae5b0 0x7f7d8809d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d880b3230 0x7f7d8809a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d88138d00 con 0x7f7d8806c6c0 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8809c540 con 0x7f7d880b3230 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8809b1f0 con 0x7f7d880ae5b0 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d8806c6c0 0x7f7d881dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d8806c6c0 0x7f7d881dd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44830/0 (socket says 172.21.15.46:44830) 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 -- 172.21.15.46:0/1713018849 learned_addr learned my addr 172.21.15.46:0/1713018849 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:37.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 -- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880ae5b0 msgr2=0x7f7d8809d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880ae5b0 0x7f7d8809d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 -- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d880b3230 msgr2=0x7f7d8809a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d880b3230 0x7f7d8809a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 -- 172.21.15.46:0/1713018849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009420 con 0x7f7d8806c6c0 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d86d9d640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d8806c6c0 0x7f7d881dd730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d780097f0 tx=0x7f7d78019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d6f7fe640 1 -- 172.21.15.46:0/1713018849 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780ac620 con 0x7f7d8806c6c0 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d6f7fe640 1 -- 172.21.15.46:0/1713018849 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d78004030 con 0x7f7d8806c6c0 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8809ca20 con 0x7f7d8806c6c0 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d6f7fe640 1 -- 172.21.15.46:0/1713018849 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780043a0 con 0x7f7d8806c6c0 2023-12-16T21:42:37.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.288+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d881dd210 con 0x7f7d8806c6c0 2023-12-16T21:42:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.292+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d880679e0 con 0x7f7d8806c6c0 2023-12-16T21:42:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.292+0000 7f7d6f7fe640 1 -- 172.21.15.46:0/1713018849 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7d780b3040 con 0x7f7d8806c6c0 2023-12-16T21:42:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.292+0000 7f7d6f7fe640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d74066470 0x7f7d74068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.292+0000 7f7d6f7fe640 1 -- 172.21.15.46:0/1713018849 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7d7811d9d0 con 0x7f7d8806c6c0 2023-12-16T21:42:37.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.292+0000 7f7d8659c640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d74066470 0x7f7d74068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.292+0000 7f7d8659c640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d74066470 0x7f7d74068930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c006aa0 tx=0x7f7d7c006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.296+0000 7f7d6f7fe640 1 -- 172.21.15.46:0/1713018849 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7d780eb230 con 0x7f7d8806c6c0 2023-12-16T21:42:37.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.484+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d881dcd90 con 0x7f7d8806c6c0 2023-12-16T21:42:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.484+0000 7f7d6f7fe640 1 -- 172.21.15.46:0/1713018849 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2114 (secure 0 0 0) 0x7f7d881dcd90 con 0x7f7d8806c6c0 2023-12-16T21:42:37.491 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-12-16T21:42:37.491 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2023-12-16T21:42:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d74066470 msgr2=0x7f7d74068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d74066470 0x7f7d74068930 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c006aa0 tx=0x7f7d7c006a30 comp rx=0 tx=0).stop 2023-12-16T21:42:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d8806c6c0 msgr2=0x7f7d881dd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d8806c6c0 0x7f7d881dd730 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d780097f0 tx=0x7f7d78019160 comp rx=0 tx=0).stop 2023-12-16T21:42:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 shutdown_connections 2023-12-16T21:42:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d74066470 0x7f7d74068930 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d880b3230 0x7f7d8809a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d880ae5b0 0x7f7d8809d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 --2- 172.21.15.46:0/1713018849 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d8806c6c0 0x7f7d881dd730 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 >> 172.21.15.46:0/1713018849 conn(0x7f7d880b48e0 msgr2=0x7f7d880b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 shutdown_connections 2023-12-16T21:42:37.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.488+0000 7f7d8d1c2640 1 -- 172.21.15.46:0/1713018849 wait complete. 2023-12-16T21:42:37.509 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.608+0000 7f0903a4f640 1 Processor -- start 2023-12-16T21:42:37.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.608+0000 7f0903a4f640 1 -- start start 2023-12-16T21:42:37.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f08fc0bc800 0x7f08fc0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08fc149c80 0x7f08fc14a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08fc14a650 con 0x7f08fc13ff50 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc140af0 con 0x7f08fc149c80 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0bcbe0 con 0x7f08fc0bc800 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08fc149c80 0x7f08fc14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f08fc149c80 0x7f08fc14a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44832/0 (socket says 172.21.15.46:44832) 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 -- 172.21.15.46:0/1675798439 learned_addr learned my addr 172.21.15.46:0/1675798439 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f09017eb640 1 --2- 172.21.15.46:0/1675798439 >> v2:172.21.15.161:3300/0 conn(0x7f08fc0bc800 0x7f08fc0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 -- 172.21.15.46:0/1675798439 >> v2:172.21.15.161:3300/0 conn(0x7f08fc0bc800 msgr2=0x7f08fc0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 --2- 172.21.15.46:0/1675798439 >> v2:172.21.15.161:3300/0 conn(0x7f08fc0bc800 0x7f08fc0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 -- 172.21.15.46:0/1675798439 >> v1:172.21.15.161:6789/0 conn(0x7f08fc13ff50 legacy=0x7f08fc1403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:37.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 -- 172.21.15.46:0/1675798439 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc0b38f0 con 0x7f08fc149c80 2023-12-16T21:42:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0900fea640 1 --2- 172.21.15.46:0/1675798439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc14a110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009870 tx=0x7f08f001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532e9ed866c41001 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f08effff640 1 -- 172.21.15.46:0/1675798439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f00aa5f0 con 0x7f08fc149c80 2023-12-16T21:42:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f08effff640 1 -- 172.21.15.46:0/1675798439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f00aa750 con 0x7f08fc149c80 2023-12-16T21:42:37.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f08effff640 1 -- 172.21.15.46:0/1675798439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f00aaae0 con 0x7f08fc149c80 2023-12-16T21:42:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- 172.21.15.46:0/1675798439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 msgr2=0x7f08fc14a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 --2- 172.21.15.46:0/1675798439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc14a110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009870 tx=0x7f08f001c490 comp rx=0 tx=0).stop 2023-12-16T21:42:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- 172.21.15.46:0/1675798439 shutdown_connections 2023-12-16T21:42:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 --2- 172.21.15.46:0/1675798439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc14a110 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 --2- 172.21.15.46:0/1675798439 >> v2:172.21.15.161:3300/0 conn(0x7f08fc0bc800 0x7f08fc0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- 172.21.15.46:0/1675798439 >> 172.21.15.46:0/1675798439 conn(0x7f08fc0722b0 msgr2=0x7f08fc0bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- 172.21.15.46:0/1675798439 shutdown_connections 2023-12-16T21:42:37.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.612+0000 7f0903a4f640 1 -- 172.21.15.46:0/1675798439 wait complete. 2023-12-16T21:42:37.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 Processor -- start 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 -- start start 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08fc0bc800 0x7f08fc13a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08fc13ff50 0x7f08fc0bb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc1de8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc13a8f0 con 0x7f08fc149c80 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc132d50 con 0x7f08fc0bc800 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc13ab70 con 0x7f08fc13ff50 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc1de8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc1de8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44848/0 (socket says 172.21.15.46:44848) 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 -- 172.21.15.46:0/312011089 learned_addr learned my addr 172.21.15.46:0/312011089 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 -- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08fc13ff50 msgr2=0x7f08fc0bb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08fc13ff50 0x7f08fc0bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 -- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08fc0bc800 msgr2=0x7f08fc13a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f09017eb640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08fc0bc800 0x7f08fc13a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08fc0bc800 0x7f08fc13a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 -- 172.21.15.46:0/312011089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f00094d0 con 0x7f08fc149c80 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0901fec640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc1de8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c780 tx=0x7f08f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f08edffb640 1 -- 172.21.15.46:0/312011089 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4004b70 con 0x7f08fc149c80 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f08edffb640 1 -- 172.21.15.46:0/312011089 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f400ce90 con 0x7f08fc149c80 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc13b9a0 con 0x7f08fc149c80 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f08edffb640 1 -- 172.21.15.46:0/312011089 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f400a850 con 0x7f08fc149c80 2023-12-16T21:42:37.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc13b6c0 con 0x7f08fc149c80 2023-12-16T21:42:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc067de0 con 0x7f08fc149c80 2023-12-16T21:42:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f08edffb640 1 -- 172.21.15.46:0/312011089 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f08f400a9b0 con 0x7f08fc149c80 2023-12-16T21:42:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.616+0000 7f08edffb640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f08e0066470 0x7f08e0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.620+0000 7f08edffb640 1 -- 172.21.15.46:0/312011089 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f08f4083c40 con 0x7f08fc149c80 2023-12-16T21:42:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.620+0000 7f09017eb640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f08e0066470 0x7f08e0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.620+0000 7f09017eb640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f08e0066470 0x7f08e0068930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08f800b800 tx=0x7f08f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.628 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.624+0000 7f08edffb640 1 -- 172.21.15.46:0/312011089 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f08f40514a0 con 0x7f08fc149c80 2023-12-16T21:42:37.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.772+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08fc13ace0 con 0x7f08fc149c80 2023-12-16T21:42:37.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.772+0000 7f08edffb640 1 -- 172.21.15.46:0/312011089 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f08fc13ace0 con 0x7f08fc149c80 2023-12-16T21:42:37.780 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:37.780 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:31.313784+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:37.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f08e0066470 msgr2=0x7f08e0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f08e0066470 0x7f08e0068930 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08f800b800 tx=0x7f08f8008040 comp rx=0 tx=0).stop 2023-12-16T21:42:37.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 msgr2=0x7f08fc1de8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc1de8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c780 tx=0x7f08f400cc40 comp rx=0 tx=0).stop 2023-12-16T21:42:37.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 shutdown_connections 2023-12-16T21:42:37.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f08e0066470 0x7f08e0068930 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f08fc149c80 0x7f08fc1de8b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f08fc13ff50 0x7f08fc0bb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 --2- 172.21.15.46:0/312011089 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f08fc0bc800 0x7f08fc13a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 >> 172.21.15.46:0/312011089 conn(0x7f08fc0722b0 msgr2=0x7f08fc146600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 shutdown_connections 2023-12-16T21:42:37.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.776+0000 7f0903a4f640 1 -- 172.21.15.46:0/312011089 wait complete. 2023-12-16T21:42:37.798 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:37.798 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.904+0000 7f2237c2a640 1 Processor -- start 2023-12-16T21:42:37.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.904+0000 7f2237c2a640 1 -- start start 2023-12-16T21:42:37.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f2237c2a640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f22300b1eb0 0x7f22300aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f2237c2a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 0x7f223014ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f2237c2a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22300b1730 con 0x7f22300af300 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f2237c2a640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f223006cd70 con 0x7f22300b1eb0 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f2237c2a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22300b0ad0 con 0x7f22301451c0 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f22300af300 0x7f2230144ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:59242/0 (socket says 172.21.15.46:59242) 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 -- 172.21.15.46:0/3198435217 learned_addr learned my addr 172.21.15.46:0/3198435217 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22359c6640 1 --2- 172.21.15.46:0/3198435217 >> v2:172.21.15.161:3300/0 conn(0x7f22300b1eb0 0x7f22300aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22361c7640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 0x7f223014ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870016128 0 0) 0x7f22300b1730 con 0x7f22300af300 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f221c003530 con 0x7f22300af300 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2818169258 0 0) 0x7f221c003530 con 0x7f22300af300 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 msgr2=0x7f223014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 0x7f223014ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 >> v2:172.21.15.161:3300/0 conn(0x7f22300b1eb0 msgr2=0x7f22300aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 --2- 172.21.15.46:0/3198435217 >> v2:172.21.15.161:3300/0 conn(0x7f22300b1eb0 0x7f22300aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.916 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223006c7f0 con 0x7f22300af300 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f2220001680 con 0x7f22300af300 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 >> v1:172.21.15.46:6789/0 conn(0x7f22300af300 legacy=0x7f2230144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c003980 0x7f221c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f221c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 0x7f221c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223006cd70 con 0x7f221c0047d0 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c00b990 con 0x7f221c003980 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221c00bb50 con 0x7f22301451c0 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f221c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22359c6640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c003980 0x7f221c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 -- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 msgr2=0x7f221c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22361c7640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 0x7f221c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.917 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301451c0 0x7f221c005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 -- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c003980 msgr2=0x7f221c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:37.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c003980 0x7f221c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22351c5640 1 -- 172.21.15.46:0/3198435217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c003530 con 0x7f221c0047d0 2023-12-16T21:42:37.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.908+0000 7f22359c6640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f221c003980 0x7f221c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:37.920 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22351c5640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f221c004c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f222000c290 tx=0x7f222001cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40e2fdd2e9e7b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200a5930 con 0x7f221c0047d0 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22200a5a90 con 0x7f221c0047d0 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22349c4640 1 -- 172.21.15.46:0/3198435217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22200a5e40 con 0x7f221c0047d0 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- 172.21.15.46:0/3198435217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 msgr2=0x7f221c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 --2- 172.21.15.46:0/3198435217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f221c004c50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f222000c290 tx=0x7f222001cce0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22361c7640 1 -- 172.21.15.46:0/3198435217 reap_dead start 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- 172.21.15.46:0/3198435217 shutdown_connections 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- 172.21.15.46:0/3198435217 >> 172.21.15.46:0/3198435217 conn(0x7f22300b4900 msgr2=0x7f22300b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- 172.21.15.46:0/3198435217 shutdown_connections 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- 172.21.15.46:0/3198435217 wait complete. 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 Processor -- start 2023-12-16T21:42:37.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- start start 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f223009c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301466f0 0x7f22301dd360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22301dd8a0 0x7f223013e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301dcf10 con 0x7f221c0047d0 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22301ddd20 con 0x7f22301dd8a0 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f2237c2a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2230146c40 con 0x7f22301466f0 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f223009c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22361c7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22301dd8a0 0x7f223013e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f223009c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44876/0 (socket says 172.21.15.46:44876) 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 -- 172.21.15.46:0/2469639219 learned_addr learned my addr 172.21.15.46:0/2469639219 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22351c5640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301466f0 0x7f22301dd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 -- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301466f0 msgr2=0x7f22301dd360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301466f0 0x7f22301dd360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 -- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22301dd8a0 msgr2=0x7f223013e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22301dd8a0 0x7f223013e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 -- 172.21.15.46:0/2469639219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223009cda0 con 0x7f221c0047d0 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.912+0000 7f22359c6640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f223009c700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f222c00f400 tx=0x7f222c00f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22267fc640 1 -- 172.21.15.46:0/2469639219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c01a070 con 0x7f221c0047d0 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22267fc640 1 -- 172.21.15.46:0/2469639219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f222c0039a0 con 0x7f221c0047d0 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22267fc640 1 -- 172.21.15.46:0/2469639219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c003d10 con 0x7f221c0047d0 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223009d080 con 0x7f221c0047d0 2023-12-16T21:42:37.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22301455d0 con 0x7f221c0047d0 2023-12-16T21:42:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2230067f70 con 0x7f221c0047d0 2023-12-16T21:42:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22267fc640 1 -- 172.21.15.46:0/2469639219 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f222c004020 con 0x7f221c0047d0 2023-12-16T21:42:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22267fc640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22140664a0 0x7f2214068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22267fc640 1 -- 172.21.15.46:0/2469639219 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f222c084580 con 0x7f221c0047d0 2023-12-16T21:42:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22351c5640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22140664a0 0x7f2214068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:37.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.916+0000 7f22351c5640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22140664a0 0x7f2214068960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22200aa000 tx=0x7f2220001a10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:37.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:37.920+0000 7f22267fc640 1 -- 172.21.15.46:0/2469639219 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f222c051e90 con 0x7f221c0047d0 2023-12-16T21:42:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.072+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2230068960 con 0x7f221c0047d0 2023-12-16T21:42:38.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.072+0000 7f22267fc640 1 -- 172.21.15.46:0/2469639219 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f222c0554b0 con 0x7f221c0047d0 2023-12-16T21:42:38.080 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22140664a0 msgr2=0x7f2214068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22140664a0 0x7f2214068960 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f22200aa000 tx=0x7f2220001a10 comp rx=0 tx=0).stop 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 msgr2=0x7f223009c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f223009c700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f222c00f400 tx=0x7f222c00f8c0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 shutdown_connections 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22140664a0 0x7f2214068960 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22301dd8a0 0x7f223013e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22301466f0 0x7f22301dd360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 --2- 172.21.15.46:0/2469639219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f221c0047d0 0x7f223009c700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 >> 172.21.15.46:0/2469639219 conn(0x7f22300b4900 msgr2=0x7f22301432e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 shutdown_connections 2023-12-16T21:42:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.076+0000 7f2237c2a640 1 -- 172.21.15.46:0/2469639219 wait complete. 2023-12-16T21:42:38.099 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-16T21:42:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe372195640 1 Processor -- start 2023-12-16T21:42:38.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe372195640 1 -- start start 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe372195640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe36c0bc800 0x7fe36c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe372195640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c149c80 0x7fe36c14a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe372195640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36c140ab0 con 0x7fe36c13ff50 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe372195640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36c14a620 con 0x7fe36c0bc800 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe372195640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c0bcbe0 con 0x7fe36c149c80 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe36c0bc800 0x7fe36c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe36c0bc800 0x7fe36c0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44880/0 (socket says 172.21.15.46:44880) 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 -- 172.21.15.46:0/218480455 learned_addr learned my addr 172.21.15.46:0/218480455 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:38.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36affd640 1 --2- 172.21.15.46:0/218480455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c149c80 0x7fe36c14a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 -- 172.21.15.46:0/218480455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c149c80 msgr2=0x7fe36c14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 --2- 172.21.15.46:0/218480455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c149c80 0x7fe36c14a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 -- 172.21.15.46:0/218480455 >> v1:172.21.15.161:6789/0 conn(0x7fe36c13ff50 legacy=0x7fe36c1403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:38.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 -- 172.21.15.46:0/218480455 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0b38f0 con 0x7fe36c0bc800 2023-12-16T21:42:38.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36b7fe640 1 --2- 172.21.15.46:0/218480455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c0bc800 0x7fe36c0b4bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009870 tx=0x7fe35c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6685af018b340c57 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:38.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36a7fc640 1 -- 172.21.15.46:0/218480455 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0aa5f0 con 0x7fe36c0bc800 2023-12-16T21:42:38.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36a7fc640 1 -- 172.21.15.46:0/218480455 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c0aa750 con 0x7fe36c0bc800 2023-12-16T21:42:38.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.220+0000 7fe36a7fc640 1 -- 172.21.15.46:0/218480455 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0aaae0 con 0x7fe36c0bc800 2023-12-16T21:42:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- 172.21.15.46:0/218480455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c0bc800 msgr2=0x7fe36c0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 --2- 172.21.15.46:0/218480455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c0bc800 0x7fe36c0b4bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009870 tx=0x7fe35c01c4d0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- 172.21.15.46:0/218480455 shutdown_connections 2023-12-16T21:42:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 --2- 172.21.15.46:0/218480455 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c149c80 0x7fe36c14a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 --2- 172.21.15.46:0/218480455 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c0bc800 0x7fe36c0b4bb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- 172.21.15.46:0/218480455 >> 172.21.15.46:0/218480455 conn(0x7fe36c0722b0 msgr2=0x7fe36c0bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:38.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- 172.21.15.46:0/218480455 shutdown_connections 2023-12-16T21:42:38.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- 172.21.15.46:0/218480455 wait complete. 2023-12-16T21:42:38.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 Processor -- start 2023-12-16T21:42:38.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- start start 2023-12-16T21:42:38.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe36c0bc800 0x7fe36c1e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c13ff50 0x7fe36c13a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c149c80 0x7fe36c148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c13de20 con 0x7fe36c149c80 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1e1c50 con 0x7fe36c0bc800 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1e1ef0 con 0x7fe36c13ff50 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c149c80 0x7fe36c148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c149c80 0x7fe36c148c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44892/0 (socket says 172.21.15.46:44892) 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 -- 172.21.15.46:0/3947047334 learned_addr learned my addr 172.21.15.46:0/3947047334 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36affd640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c13ff50 0x7fe36c13a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36b7fe640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe36c0bc800 0x7fe36c1e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 -- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c13ff50 msgr2=0x7fe36c13a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c13ff50 0x7fe36c13a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 -- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe36c0bc800 msgr2=0x7fe36c1e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe36c0bc800 0x7fe36c1e26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 -- 172.21.15.46:0/3947047334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0094d0 con 0x7fe36c149c80 2023-12-16T21:42:38.233 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe36bfff640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c149c80 0x7fe36c148c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c780 tx=0x7fe36400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:38.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe353fff640 1 -- 172.21.15.46:0/3947047334 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe364004b70 con 0x7fe36c149c80 2023-12-16T21:42:38.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe353fff640 1 -- 172.21.15.46:0/3947047334 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36400ce90 con 0x7fe36c149c80 2023-12-16T21:42:38.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c1e21d0 con 0x7fe36c149c80 2023-12-16T21:42:38.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe353fff640 1 -- 172.21.15.46:0/3947047334 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36400a850 con 0x7fe36c149c80 2023-12-16T21:42:38.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.224+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c0b4bb0 con 0x7fe36c149c80 2023-12-16T21:42:38.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.228+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c067de0 con 0x7fe36c149c80 2023-12-16T21:42:38.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.228+0000 7fe353fff640 1 -- 172.21.15.46:0/3947047334 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fe36400a9b0 con 0x7fe36c149c80 2023-12-16T21:42:38.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.228+0000 7fe353fff640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe358066470 0x7fe358068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.228+0000 7fe353fff640 1 -- 172.21.15.46:0/3947047334 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe364083d20 con 0x7fe36c149c80 2023-12-16T21:42:38.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.228+0000 7fe36b7fe640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe358066470 0x7fe358068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.228+0000 7fe36b7fe640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe358066470 0x7fe358068930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0ae000 tx=0x7fe35c0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:38.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.232+0000 7fe353fff640 1 -- 172.21.15.46:0/3947047334 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fe3640514e0 con 0x7fe36c149c80 2023-12-16T21:42:38.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe36c1e2410 con 0x7fe36c149c80 2023-12-16T21:42:38.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe353fff640 1 -- 172.21.15.46:0/3947047334 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe36c1e2410 con 0x7fe36c149c80 2023-12-16T21:42:38.398 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:38.398 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-12-16T21:42:38.398 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:42:38.398 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-12-16T21:42:38.398 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-12-16T21:42:38.398 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:42:38.399 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-12-16T21:42:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe358066470 msgr2=0x7fe358068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe358066470 0x7fe358068930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0ae000 tx=0x7fe35c0a7000 comp rx=0 tx=0).stop 2023-12-16T21:42:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c149c80 msgr2=0x7fe36c148c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c149c80 0x7fe36c148c10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe36400c780 tx=0x7fe36400cc40 comp rx=0 tx=0).stop 2023-12-16T21:42:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 shutdown_connections 2023-12-16T21:42:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe358066470 0x7fe358068930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe36c149c80 0x7fe36c148c10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe36c13ff50 0x7fe36c13a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 --2- 172.21.15.46:0/3947047334 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe36c0bc800 0x7fe36c1e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 >> 172.21.15.46:0/3947047334 conn(0x7fe36c0722b0 msgr2=0x7fe36c144600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:38.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 shutdown_connections 2023-12-16T21:42:38.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.392+0000 7fe372195640 1 -- 172.21.15.46:0/3947047334 wait complete. 2023-12-16T21:42:38.417 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-16T21:42:38.417 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-16T21:42:38.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5288785640 1 Processor -- start 2023-12-16T21:42:38.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5288785640 1 -- start start 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5288785640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52800b29f0 0x7f52800b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5288785640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f52800ae0f0 0x7f52800ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5288785640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5280072550 con 0x7f52800b0fb0 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5288785640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52800ae750 con 0x7f52800b29f0 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5288785640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52800b2dd0 con 0x7f52800ae0f0 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286521640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52800b29f0 0x7f52800b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5285d20640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f52800b0fb0 0x7f52800acdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42966/0 (socket says 172.21.15.46:42966) 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286521640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f52800b29f0 0x7f52800b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44732/0 (socket says 172.21.15.46:44732) 2023-12-16T21:42:38.543 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5285d20640 1 -- 172.21.15.46:0/3746356122 learned_addr learned my addr 172.21.15.46:0/3746356122 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286521640 1 -- 172.21.15.46:0/3746356122 >> v2:172.21.15.161:3300/0 conn(0x7f52800ae0f0 msgr2=0x7f52800ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286d22640 1 --2- 172.21.15.46:0/3746356122 >> v2:172.21.15.161:3300/0 conn(0x7f52800ae0f0 0x7f52800ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286521640 1 --2- 172.21.15.46:0/3746356122 >> v2:172.21.15.161:3300/0 conn(0x7f52800ae0f0 0x7f52800ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286521640 1 -- 172.21.15.46:0/3746356122 >> v1:172.21.15.46:6789/0 conn(0x7f52800b0fb0 legacy=0x7f52800acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f528551f640 1 -- 172.21.15.46:0/3746356122 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800264397 0 0) 0x7f5274001120 con 0x7f52800b0fb0 2023-12-16T21:42:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286521640 1 -- 172.21.15.46:0/3746356122 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52800b0300 con 0x7f52800b29f0 2023-12-16T21:42:38.544 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.536+0000 7f5286d22640 1 --2- 172.21.15.46:0/3746356122 >> v2:172.21.15.161:3300/0 conn(0x7f52800ae0f0 0x7f52800ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:38.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5286521640 1 --2- 172.21.15.46:0/3746356122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52800b0a70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52800af670 tx=0x7f527801e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e09f70e284ff149 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:38.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f528551f640 1 -- 172.21.15.46:0/3746356122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527800b2d0 con 0x7f52800b29f0 2023-12-16T21:42:38.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f528551f640 1 -- 172.21.15.46:0/3746356122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52780a7790 con 0x7f52800b29f0 2023-12-16T21:42:38.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f528551f640 1 -- 172.21.15.46:0/3746356122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52780a7b20 con 0x7f52800b29f0 2023-12-16T21:42:38.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- 172.21.15.46:0/3746356122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 msgr2=0x7f52800b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 --2- 172.21.15.46:0/3746356122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52800b0a70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f52800af670 tx=0x7f527801e9c0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- 172.21.15.46:0/3746356122 shutdown_connections 2023-12-16T21:42:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 --2- 172.21.15.46:0/3746356122 >> v2:172.21.15.161:3300/0 conn(0x7f52800ae0f0 0x7f52800ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 --2- 172.21.15.46:0/3746356122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52800b0a70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- 172.21.15.46:0/3746356122 >> 172.21.15.46:0/3746356122 conn(0x7f52800b4890 msgr2=0x7f52800b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- 172.21.15.46:0/3746356122 shutdown_connections 2023-12-16T21:42:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- 172.21.15.46:0/3746356122 wait complete. 2023-12-16T21:42:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 Processor -- start 2023-12-16T21:42:38.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- start start 2023-12-16T21:42:38.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52800b0fb0 0x7f5280144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52801e0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f52801e0540 0x7f528014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52801e2cd0 con 0x7f52800b29f0 2023-12-16T21:42:38.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52801e30f0 con 0x7f52800b0fb0 2023-12-16T21:42:38.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5288785640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528009aa30 con 0x7f52801e0540 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5285d20640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52801e0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5285d20640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52801e0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44738/0 (socket says 172.21.15.46:44738) 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5285d20640 1 -- 172.21.15.46:0/3759678949 learned_addr learned my addr 172.21.15.46:0/3759678949 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5286d22640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f52801e0540 0x7f528014c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.540+0000 7f5286521640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52800b0fb0 0x7f5280144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5285d20640 1 -- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f52801e0540 msgr2=0x7f528014c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5285d20640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f52801e0540 0x7f528014c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5285d20640 1 -- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52800b0fb0 msgr2=0x7f5280144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5285d20640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52800b0fb0 0x7f5280144f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5285d20640 1 -- 172.21.15.46:0/3759678949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527800bc80 con 0x7f52800b29f0 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5286521640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52800b0fb0 0x7f5280144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5285d20640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52801e0000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f527400c7b0 tx=0x7f527400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f527effd640 1 -- 172.21.15.46:0/3759678949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52740072d0 con 0x7f52800b29f0 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f527effd640 1 -- 172.21.15.46:0/3759678949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5274007430 con 0x7f52800b29f0 2023-12-16T21:42:38.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52800af000 con 0x7f52800b29f0 2023-12-16T21:42:38.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f527effd640 1 -- 172.21.15.46:0/3759678949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274001d20 con 0x7f52800b29f0 2023-12-16T21:42:38.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52801e25e0 con 0x7f52800b29f0 2023-12-16T21:42:38.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52801e2e10 con 0x7f52800b29f0 2023-12-16T21:42:38.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f527effd640 1 -- 172.21.15.46:0/3759678949 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f52740047f0 con 0x7f52800b29f0 2023-12-16T21:42:38.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f527effd640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f526c0664a0 0x7f526c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:38.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f527effd640 1 -- 172.21.15.46:0/3759678949 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5274082e60 con 0x7f52800b29f0 2023-12-16T21:42:38.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5286521640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f526c0664a0 0x7f526c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:38.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.544+0000 7f5286521640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f526c0664a0 0x7f526c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52780b0000 tx=0x7f52780a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:38.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.548+0000 7f527effd640 1 -- 172.21.15.46:0/3759678949 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f52801e2e10 con 0x7f52800b29f0 2023-12-16T21:42:38.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:38.700+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f52801e2e10 con 0x7f52800b29f0 2023-12-16T21:42:39.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.379+0000 7f527effd640 1 -- 172.21.15.46:0/3759678949 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52801e2e10 con 0x7f52800b29f0 2023-12-16T21:42:39.386 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-12-16T21:42:39.389 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f526c0664a0 msgr2=0x7f526c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f526c0664a0 0x7f526c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52780b0000 tx=0x7f52780a9070 comp rx=0 tx=0).stop 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 msgr2=0x7f52801e0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52801e0000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f527400c7b0 tx=0x7f527400cc70 comp rx=0 tx=0).stop 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 shutdown_connections 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f526c0664a0 0x7f526c068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f52801e0540 0x7f528014c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f52800b29f0 0x7f52801e0000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 --2- 172.21.15.46:0/3759678949 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f52800b0fb0 0x7f5280144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 >> 172.21.15.46:0/3759678949 conn(0x7f52800b4890 msgr2=0x7f52800b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 shutdown_connections 2023-12-16T21:42:39.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.383+0000 7f5288785640 1 -- 172.21.15.46:0/3759678949 wait complete. 2023-12-16T21:42:39.416 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-16T21:42:39.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 Processor -- start 2023-12-16T21:42:39.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 -- start start 2023-12-16T21:42:39.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e8406dd80 0x7f5e8406e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:39.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f5e84071bf0 0x7f5e841443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:39.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e841415e0 0x7f5e84141a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:39.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e840723f0 con 0x7f5e8406dd80 2023-12-16T21:42:39.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e8406e6a0 con 0x7f5e84071bf0 2023-12-16T21:42:39.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8c34b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84066c60 con 0x7f5e841415e0 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e8406dd80 0x7f5e8406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f5e8406dd80 0x7f5e8406e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44752/0 (socket says 172.21.15.46:44752) 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/1044739102 learned_addr learned my addr 172.21.15.46:0/1044739102 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e898e6640 1 --2- 172.21.15.46:0/1044739102 >> v2:172.21.15.161:3300/0 conn(0x7f5e84071bf0 0x7f5e841443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a8e8640 1 --2- 172.21.15.46:0/1044739102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e841415e0 0x7f5e84141a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/1044739102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e841415e0 msgr2=0x7f5e84141a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 --2- 172.21.15.46:0/1044739102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e841415e0 0x7f5e84141a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/1044739102 >> v2:172.21.15.161:3300/0 conn(0x7f5e84071bf0 msgr2=0x7f5e841443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 --2- 172.21.15.46:0/1044739102 >> v2:172.21.15.161:3300/0 conn(0x7f5e84071bf0 0x7f5e841443e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.539+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/1044739102 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e840713f0 con 0x7f5e8406dd80 2023-12-16T21:42:39.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8a0e7640 1 --2- 172.21.15.46:0/1044739102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e8406e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e84068120 tx=0x7f5e8001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab5f4d254e3ed2e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:39.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e890e5640 1 -- 172.21.15.46:0/1044739102 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800ac630 con 0x7f5e8406dd80 2023-12-16T21:42:39.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e890e5640 1 -- 172.21.15.46:0/1044739102 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e800ac790 con 0x7f5e8406dd80 2023-12-16T21:42:39.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e890e5640 1 -- 172.21.15.46:0/1044739102 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800acb20 con 0x7f5e8406dd80 2023-12-16T21:42:39.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 -- 172.21.15.46:0/1044739102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 msgr2=0x7f5e8406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:39.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/1044739102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e8406e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e84068120 tx=0x7f5e8001c500 comp rx=0 tx=0).stop 2023-12-16T21:42:39.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 -- 172.21.15.46:0/1044739102 shutdown_connections 2023-12-16T21:42:39.551 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/1044739102 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e841415e0 0x7f5e84141a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/1044739102 >> v2:172.21.15.161:3300/0 conn(0x7f5e84071bf0 0x7f5e841443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/1044739102 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e8406e160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 -- 172.21.15.46:0/1044739102 >> 172.21.15.46:0/1044739102 conn(0x7f5e840b6df0 msgr2=0x7f5e840b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 -- 172.21.15.46:0/1044739102 shutdown_connections 2023-12-16T21:42:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 -- 172.21.15.46:0/1044739102 wait complete. 2023-12-16T21:42:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 Processor -- start 2023-12-16T21:42:39.552 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.543+0000 7f5e8c34b640 1 -- start start 2023-12-16T21:42:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e84139120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e84071bf0 0x7f5e841df930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e841415e0 0x7f5e84137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:39.554 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8413ce50 con 0x7f5e8406dd80 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8413d7c0 con 0x7f5e841415e0 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84143d70 con 0x7f5e84071bf0 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e84139120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e84139120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44762/0 (socket says 172.21.15.46:44762) 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/950003499 learned_addr learned my addr 172.21.15.46:0/950003499 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e84071bf0 msgr2=0x7f5e841df930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e898e6640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e84071bf0 0x7f5e841df930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a8e8640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e841415e0 0x7f5e84137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e84071bf0 0x7f5e841df930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e841415e0 msgr2=0x7f5e84137080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e841415e0 0x7f5e84137080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 -- 172.21.15.46:0/950003499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e800094d0 con 0x7f5e8406dd80 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a8e8640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e841415e0 0x7f5e84137080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:39.555 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8a0e7640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e84139120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e8001cda0 tx=0x7f5e800192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:39.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e7affd640 1 -- 172.21.15.46:0/950003499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800b5640 con 0x7f5e8406dd80 2023-12-16T21:42:39.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e7affd640 1 -- 172.21.15.46:0/950003499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e800b57a0 con 0x7f5e8406dd80 2023-12-16T21:42:39.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e7affd640 1 -- 172.21.15.46:0/950003499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e800b5b30 con 0x7f5e8406dd80 2023-12-16T21:42:39.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8413da80 con 0x7f5e8406dd80 2023-12-16T21:42:39.556 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e84143a90 con 0x7f5e8406dd80 2023-12-16T21:42:39.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e84068960 con 0x7f5e8406dd80 2023-12-16T21:42:39.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e7affd640 1 -- 172.21.15.46:0/950003499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f5e800b3070 con 0x7f5e8406dd80 2023-12-16T21:42:39.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e7affd640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5e680664a0 0x7f5e68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:39.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e7affd640 1 -- 172.21.15.46:0/950003499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5e8011dd00 con 0x7f5e8406dd80 2023-12-16T21:42:39.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.547+0000 7f5e898e6640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5e680664a0 0x7f5e68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:39.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.551+0000 7f5e898e6640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5e680664a0 0x7f5e68068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e84067f50 tx=0x7f5e74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:39.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.555+0000 7f5e7affd640 1 -- 172.21.15.46:0/950003499 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f5e800eb3e0 con 0x7f5e8406dd80 2023-12-16T21:42:39.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:39.703+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5e8413d050 con 0x7f5e8406dd80 2023-12-16T21:42:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.391+0000 7f5e7affd640 1 -- 172.21.15.46:0/950003499 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5e800eea00 con 0x7f5e8406dd80 2023-12-16T21:42:40.399 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5e680664a0 msgr2=0x7f5e68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5e680664a0 0x7f5e68068960 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5e84067f50 tx=0x7f5e74008040 comp rx=0 tx=0).stop 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 msgr2=0x7f5e84139120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e84139120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e8001cda0 tx=0x7f5e800192a0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 shutdown_connections 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5e680664a0 0x7f5e68068960 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5e841415e0 0x7f5e84137080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5e84071bf0 0x7f5e841df930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 --2- 172.21.15.46:0/950003499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5e8406dd80 0x7f5e84139120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 >> 172.21.15.46:0/950003499 conn(0x7f5e840b6df0 msgr2=0x7f5e8413a920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 shutdown_connections 2023-12-16T21:42:40.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.395+0000 7f5e8c34b640 1 -- 172.21.15.46:0/950003499 wait complete. 2023-12-16T21:42:40.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-16T21:42:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015f5e9640 1 Processor -- start 2023-12-16T21:42:40.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015f5e9640 1 -- start start 2023-12-16T21:42:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015f5e9640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f015806c8b0 0x7f01580b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015f5e9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01580ae750 0x7f01580ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015f5e9640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01580b02e0 con 0x7f01580af300 2023-12-16T21:42:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015f5e9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01580aded0 con 0x7f01580ae750 2023-12-16T21:42:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015f5e9640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015806cc90 con 0x7f015806c8b0 2023-12-16T21:42:40.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015db86640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01580ae750 0x7f01580ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015db86640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f01580ae750 0x7f01580ad530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44774/0 (socket says 172.21.15.46:44774) 2023-12-16T21:42:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015db86640 1 -- 172.21.15.46:0/1932406610 learned_addr learned my addr 172.21.15.46:0/1932406610 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015d385640 1 --2- 172.21.15.46:0/1932406610 >> v2:172.21.15.161:3300/0 conn(0x7f015806c8b0 0x7f01580b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015db86640 1 -- 172.21.15.46:0/1932406610 >> v2:172.21.15.161:3300/0 conn(0x7f015806c8b0 msgr2=0x7f01580b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015db86640 1 --2- 172.21.15.46:0/1932406610 >> v2:172.21.15.161:3300/0 conn(0x7f015806c8b0 0x7f01580b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015db86640 1 -- 172.21.15.46:0/1932406610 >> v1:172.21.15.161:6789/0 conn(0x7f01580af300 legacy=0x7f01580ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:40.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.563+0000 7f015db86640 1 -- 172.21.15.46:0/1932406610 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01580b05c0 con 0x7f01580ae750 2023-12-16T21:42:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 --2- 172.21.15.46:0/1932406610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580ae750 0x7f01580ad530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01580b3c90 tx=0x7f015001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a304f86b2a914456 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f014ffff640 1 -- 172.21.15.46:0/1932406610 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500a5790 con 0x7f01580ae750 2023-12-16T21:42:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f014ffff640 1 -- 172.21.15.46:0/1932406610 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01500a58f0 con 0x7f01580ae750 2023-12-16T21:42:40.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f014ffff640 1 -- 172.21.15.46:0/1932406610 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500a5c80 con 0x7f01580ae750 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- 172.21.15.46:0/1932406610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580ae750 msgr2=0x7f01580ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 --2- 172.21.15.46:0/1932406610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580ae750 0x7f01580ad530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f01580b3c90 tx=0x7f015001c8d0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- 172.21.15.46:0/1932406610 shutdown_connections 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 --2- 172.21.15.46:0/1932406610 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580ae750 0x7f01580ad530 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 --2- 172.21.15.46:0/1932406610 >> v2:172.21.15.161:3300/0 conn(0x7f015806c8b0 0x7f01580b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- 172.21.15.46:0/1932406610 >> 172.21.15.46:0/1932406610 conn(0x7f01580b4900 msgr2=0x7f01580b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- 172.21.15.46:0/1932406610 shutdown_connections 2023-12-16T21:42:40.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- 172.21.15.46:0/1932406610 wait complete. 2023-12-16T21:42:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 Processor -- start 2023-12-16T21:42:40.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- start start 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015806c8b0 0x7f01580b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01580ae750 0x7f0158141b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580af300 0x7f01581dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0158142080 con 0x7f01580af300 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015813be50 con 0x7f01580ae750 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015f5e9640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01580b32e0 con 0x7f015806c8b0 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580af300 0x7f01581dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580af300 0x7f01581dfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44782/0 (socket says 172.21.15.46:44782) 2023-12-16T21:42:40.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 -- 172.21.15.46:0/3542599135 learned_addr learned my addr 172.21.15.46:0/3542599135 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015d385640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015806c8b0 0x7f01580b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015cb84640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01580ae750 0x7f0158141b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 -- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015806c8b0 msgr2=0x7f01580b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015806c8b0 0x7f01580b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 -- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01580ae750 msgr2=0x7f0158141b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01580ae750 0x7f0158141b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 -- 172.21.15.46:0/3542599135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0150009b30 con 0x7f01580af300 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.567+0000 7f015db86640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580af300 0x7f01581dfed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f015001a500 tx=0x7f0150019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f014dffb640 1 -- 172.21.15.46:0/3542599135 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500b8670 con 0x7f01580af300 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f014dffb640 1 -- 172.21.15.46:0/3542599135 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01500b87f0 con 0x7f01580af300 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01581467b0 con 0x7f01580af300 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f014dffb640 1 -- 172.21.15.46:0/3542599135 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01500b8ba0 con 0x7f01580af300 2023-12-16T21:42:40.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0158146480 con 0x7f01580af300 2023-12-16T21:42:40.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0158068960 con 0x7f01580af300 2023-12-16T21:42:40.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f014dffb640 1 -- 172.21.15.46:0/3542599135 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f01500b8d00 con 0x7f01580af300 2023-12-16T21:42:40.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f014dffb640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f013c066470 0x7f013c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:40.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f014dffb640 1 -- 172.21.15.46:0/3542599135 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f015011f6c0 con 0x7f01580af300 2023-12-16T21:42:40.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f015d385640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f013c066470 0x7f013c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:40.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.571+0000 7f015d385640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f013c066470 0x7f013c068930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f015400a970 tx=0x7f0154008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:40.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.575+0000 7f014dffb640 1 -- 172.21.15.46:0/3542599135 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f01500ecd50 con 0x7f01580af300 2023-12-16T21:42:40.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:40.771+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f01580b0ea0 con 0x7f01580af300 2023-12-16T21:42:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.403+0000 7f014dffb640 1 -- 172.21.15.46:0/3542599135 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f01500f0370 con 0x7f01580af300 2023-12-16T21:42:41.409 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2023-12-16T21:42:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.403+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f013c066470 msgr2=0x7f013c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f013c066470 0x7f013c068930 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f015400a970 tx=0x7f0154008040 comp rx=0 tx=0).stop 2023-12-16T21:42:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580af300 msgr2=0x7f01581dfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580af300 0x7f01581dfed0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f015001a500 tx=0x7f0150019670 comp rx=0 tx=0).stop 2023-12-16T21:42:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 shutdown_connections 2023-12-16T21:42:41.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f013c066470 0x7f013c068930 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01580af300 0x7f01581dfed0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f01580ae750 0x7f0158141b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 --2- 172.21.15.46:0/3542599135 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f015806c8b0 0x7f01580b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 >> 172.21.15.46:0/3542599135 conn(0x7f01580b4900 msgr2=0x7f0158070a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:41.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 shutdown_connections 2023-12-16T21:42:41.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.407+0000 7f015f5e9640 1 -- 172.21.15.46:0/3542599135 wait complete. 2023-12-16T21:42:41.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.555+0000 7f8de2cde640 1 Processor -- start 2023-12-16T21:42:41.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.555+0000 7f8de2cde640 1 -- start start 2023-12-16T21:42:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de2cde640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8ddc0b0340 0x7f8ddc13ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de2cde640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0af2e0 con 0x7f8ddc06c920 2023-12-16T21:42:41.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de2cde640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc0b1f50 con 0x7f8ddc0b3fd0 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de2cde640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc06cd00 con 0x7f8ddc0b0340 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de0a7a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ddc06c920 0x7f8ddc0afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42968/0 (socket says 172.21.15.46:42968) 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de0a7a640 1 -- 172.21.15.46:0/3749159182 learned_addr learned my addr 172.21.15.46:0/3749159182 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de127b640 1 --2- 172.21.15.46:0/3749159182 >> v2:172.21.15.161:3300/0 conn(0x7f8ddc0b0340 0x7f8ddc13ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155259269 0 0) 0x7f8ddc0af2e0 con 0x7f8ddc06c920 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc8003530 con 0x7f8ddc06c920 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963675656 0 0) 0x7f8dc8003530 con 0x7f8ddc06c920 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 >> v2:172.21.15.161:3300/0 conn(0x7f8ddc0b0340 msgr2=0x7f8ddc13ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:41.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 --2- 172.21.15.46:0/3749159182 >> v2:172.21.15.161:3300/0 conn(0x7f8ddc0b0340 0x7f8ddc13ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 >> v1:172.21.15.161:6789/0 conn(0x7f8ddc0b3fd0 legacy=0x7f8ddc13e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0af530 con 0x7f8ddc06c920 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f8dd4001680 con 0x7f8ddc06c920 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 >> v1:172.21.15.46:6789/0 conn(0x7f8ddc06c920 legacy=0x7f8ddc0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dc8003980 0x7f8dc8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8dc8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dc8005190 0x7f8dc800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc06cd00 con 0x7f8dc80047d0 2023-12-16T21:42:41.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc800c130 con 0x7f8dc8005190 2023-12-16T21:42:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc800c2f0 con 0x7f8dc8003980 2023-12-16T21:42:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddbfff640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8dc8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de0a7a640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dc8003980 0x7f8dc8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de127b640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dc8005190 0x7f8dc800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddbfff640 1 -- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dc8003980 msgr2=0x7f8dc8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddbfff640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dc8003980 0x7f8dc8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddbfff640 1 -- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dc8005190 msgr2=0x7f8dc800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddbfff640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dc8005190 0x7f8dc800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8ddbfff640 1 -- 172.21.15.46:0/3749159182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8003530 con 0x7f8dc80047d0 2023-12-16T21:42:41.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.559+0000 7f8de127b640 1 -- 172.21.15.46:0/3749159182 reap_dead start 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8ddbfff640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8dc8004c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4001e70 tx=0x7f8dc401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80aa0deb36499673 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4003b80 con 0x7f8dc80047d0 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4003ce0 con 0x7f8dc80047d0 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8ddb7fe640 1 -- 172.21.15.46:0/3749159182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4005310 con 0x7f8dc80047d0 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- 172.21.15.46:0/3749159182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 msgr2=0x7f8dc8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8dc8004c50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4001e70 tx=0x7f8dc401c740 comp rx=0 tx=0).stop 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- 172.21.15.46:0/3749159182 shutdown_connections 2023-12-16T21:42:41.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3749159182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8dc8004c50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- 172.21.15.46:0/3749159182 >> 172.21.15.46:0/3749159182 conn(0x7f8ddc0b48e0 msgr2=0x7f8ddc0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- 172.21.15.46:0/3749159182 shutdown_connections 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- 172.21.15.46:0/3749159182 wait complete. 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 Processor -- start 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- start start 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8ddc147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ddc06ed90 0x7f8ddc1e0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc1e0aa0 0x7f8ddc139f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc06f210 con 0x7f8dc80047d0 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1e0f20 con 0x7f8ddc1e0aa0 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc0704c0 con 0x7f8ddc06ed90 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8ddc147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8ddc147d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44796/0 (socket says 172.21.15.46:44796) 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 -- 172.21.15.46:0/3025291765 learned_addr learned my addr 172.21.15.46:0/3025291765 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8ddbfff640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ddc06ed90 0x7f8ddc1e0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de127b640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc1e0aa0 0x7f8ddc139f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 -- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ddc06ed90 msgr2=0x7f8ddc1e0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ddc06ed90 0x7f8ddc1e0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 -- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc1e0aa0 msgr2=0x7f8ddc139f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc1e0aa0 0x7f8ddc139f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 -- 172.21.15.46:0/3025291765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc0707a0 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de0a7a640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8ddc147d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400e780 tx=0x7f8dd400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8dd97fa640 1 -- 172.21.15.46:0/3025291765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400ee30 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8dd97fa640 1 -- 172.21.15.46:0/3025291765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd400c8d0 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc148340 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.563+0000 7f8dd97fa640 1 -- 172.21.15.46:0/3025291765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd400cc80 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.567+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc148900 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.567+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ddc0679e0 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.567+0000 7f8dd97fa640 1 -- 172.21.15.46:0/3025291765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8dd4013420 con 0x7f8dc80047d0 2023-12-16T21:42:41.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.567+0000 7f8dd97fa640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dbc0664a0 0x7f8dbc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.567+0000 7f8dd97fa640 1 -- 172.21.15.46:0/3025291765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dd4083a80 con 0x7f8dc80047d0 2023-12-16T21:42:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.567+0000 7f8ddbfff640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dbc0664a0 0x7f8dbc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.567+0000 7f8ddbfff640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dbc0664a0 0x7f8dbc068960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40ad000 tx=0x7f8dc4001520 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:41.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.571+0000 7f8dd97fa640 1 -- 172.21.15.46:0/3025291765 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8dd4050f00 con 0x7f8dc80047d0 2023-12-16T21:42:41.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.727+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ddc0688f0 con 0x7f8dc80047d0 2023-12-16T21:42:41.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8dd97fa640 1 -- 172.21.15.46:0/3025291765 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16518 (secure 0 0 0) 0x7f8dd4054520 con 0x7f8dc80047d0 2023-12-16T21:42:41.738 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:41.739 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:40.776061+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:41.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dbc0664a0 msgr2=0x7f8dbc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dbc0664a0 0x7f8dbc068960 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8dc40ad000 tx=0x7f8dc4001520 comp rx=0 tx=0).stop 2023-12-16T21:42:41.740 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 msgr2=0x7f8ddc147d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8ddc147d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400e780 tx=0x7f8dd400ec40 comp rx=0 tx=0).stop 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 shutdown_connections 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dbc0664a0 0x7f8dbc068960 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ddc1e0aa0 0x7f8ddc139f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ddc06ed90 0x7f8ddc1e0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 --2- 172.21.15.46:0/3025291765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dc80047d0 0x7f8ddc147d60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 >> 172.21.15.46:0/3025291765 conn(0x7f8ddc0b48e0 msgr2=0x7f8ddc0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.731+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 shutdown_connections 2023-12-16T21:42:41.741 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.735+0000 7f8de2cde640 1 -- 172.21.15.46:0/3025291765 wait complete. 2023-12-16T21:42:41.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:41.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f143e1e7640 1 Processor -- start 2023-12-16T21:42:41.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f143e1e7640 1 -- start start 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f143e1e7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f143814a390 0x7f14380b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f143e1e7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380bcbe0 con 0x7f143813ff50 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f143e1e7640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380b38f0 con 0x7f14380bc800 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f143e1e7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1438140ad0 con 0x7f143814a390 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f1436ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f143814a390 0x7f14380b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f1436ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f143814a390 0x7f14380b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44812/0 (socket says 172.21.15.46:44812) 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f1436ffd640 1 -- 172.21.15.46:0/4138809565 learned_addr learned my addr 172.21.15.46:0/4138809565 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f1436ffd640 1 -- 172.21.15.46:0/4138809565 >> v1:172.21.15.161:6789/0 conn(0x7f14380bc800 legacy=0x7f1438149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f1436ffd640 1 -- 172.21.15.46:0/4138809565 >> v1:172.21.15.46:6789/0 conn(0x7f143813ff50 legacy=0x7f14381403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f1436ffd640 1 -- 172.21.15.46:0/4138809565 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380b3bd0 con 0x7f143814a390 2023-12-16T21:42:41.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.875+0000 7f14367fc640 1 -- 172.21.15.46:0/4138809565 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515073419 0 0) 0x7f14380bcbe0 con 0x7f143813ff50 2023-12-16T21:42:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f1436ffd640 1 --2- 172.21.15.46:0/4138809565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143814a390 0x7f14380b4bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f142c009870 tx=0x7f142c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a762c3566c847364 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f14367fc640 1 -- 172.21.15.46:0/4138809565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0ab620 con 0x7f143814a390 2023-12-16T21:42:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f14367fc640 1 -- 172.21.15.46:0/4138809565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c0ab780 con 0x7f143814a390 2023-12-16T21:42:41.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f14367fc640 1 -- 172.21.15.46:0/4138809565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0abb10 con 0x7f143814a390 2023-12-16T21:42:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 -- 172.21.15.46:0/4138809565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143814a390 msgr2=0x7f14380b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 --2- 172.21.15.46:0/4138809565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143814a390 0x7f14380b4bb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f142c009870 tx=0x7f142c01c510 comp rx=0 tx=0).stop 2023-12-16T21:42:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 -- 172.21.15.46:0/4138809565 shutdown_connections 2023-12-16T21:42:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 --2- 172.21.15.46:0/4138809565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143814a390 0x7f14380b4bb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 -- 172.21.15.46:0/4138809565 >> 172.21.15.46:0/4138809565 conn(0x7f14380722b0 msgr2=0x7f14380bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:41.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 -- 172.21.15.46:0/4138809565 shutdown_connections 2023-12-16T21:42:41.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 -- 172.21.15.46:0/4138809565 wait complete. 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 Processor -- start 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.879+0000 7f143e1e7640 1 -- start start 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14380bc800 0x7f14381df770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143813ff50 0x7f1438148680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f143814a390 0x7f14381e2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143813c2e0 con 0x7f143813ff50 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143813c000 con 0x7f143814a390 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14381decb0 con 0x7f14380bc800 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143813ff50 0x7f1438148680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143813ff50 0x7f1438148680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44820/0 (socket says 172.21.15.46:44820) 2023-12-16T21:42:41.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 -- 172.21.15.46:0/1141667782 learned_addr learned my addr 172.21.15.46:0/1141667782 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1437fff640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f143814a390 0x7f14381e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f14377fe640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14380bc800 0x7f14381df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 -- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14380bc800 msgr2=0x7f14381df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14380bc800 0x7f14381df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 -- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f143814a390 msgr2=0x7f14381e2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f143814a390 0x7f14381e2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 -- 172.21.15.46:0/1141667782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0094d0 con 0x7f143813ff50 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1437fff640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f143814a390 0x7f14381e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f1436ffd640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143813ff50 0x7f1438148680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f142c0ad040 tx=0x7f142c01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f141ffff640 1 -- 172.21.15.46:0/1141667782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0be3f0 con 0x7f143813ff50 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f141ffff640 1 -- 172.21.15.46:0/1141667782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c0be570 con 0x7f143813ff50 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f141ffff640 1 -- 172.21.15.46:0/1141667782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0be920 con 0x7f143813ff50 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14381def90 con 0x7f143813ff50 2023-12-16T21:42:41.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14381df320 con 0x7f143813ff50 2023-12-16T21:42:41.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1438067de0 con 0x7f143813ff50 2023-12-16T21:42:41.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f141ffff640 1 -- 172.21.15.46:0/1141667782 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f142c0b43f0 con 0x7f143813ff50 2023-12-16T21:42:41.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f141ffff640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1424066470 0x7f1424068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f141ffff640 1 -- 172.21.15.46:0/1141667782 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f142c11e0d0 con 0x7f143813ff50 2023-12-16T21:42:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.883+0000 7f14377fe640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1424066470 0x7f1424068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:41.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.887+0000 7f14377fe640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1424066470 0x7f1424068930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1428000f70 tx=0x7f1428009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:41.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:41.887+0000 7f141ffff640 1 -- 172.21.15.46:0/1141667782 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f142c0eb690 con 0x7f143813ff50 2023-12-16T21:42:42.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.047+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14380688f0 con 0x7f143813ff50 2023-12-16T21:42:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.047+0000 7f141ffff640 1 -- 172.21.15.46:0/1141667782 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16518 (secure 0 0 0) 0x7f142c0eecb0 con 0x7f143813ff50 2023-12-16T21:42:42.055 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:42.055 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":20,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:40.776061+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1424066470 msgr2=0x7f1424068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1424066470 0x7f1424068930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1428000f70 tx=0x7f1428009040 comp rx=0 tx=0).stop 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143813ff50 msgr2=0x7f1438148680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143813ff50 0x7f1438148680 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f142c0ad040 tx=0x7f142c01ad50 comp rx=0 tx=0).stop 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 shutdown_connections 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1424066470 0x7f1424068930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f143814a390 0x7f14381e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f143813ff50 0x7f1438148680 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 --2- 172.21.15.46:0/1141667782 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14380bc800 0x7f14381df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 >> 172.21.15.46:0/1141667782 conn(0x7f14380722b0 msgr2=0x7f14380b9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 shutdown_connections 2023-12-16T21:42:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.051+0000 7f143e1e7640 1 -- 172.21.15.46:0/1141667782 wait complete. 2023-12-16T21:42:42.075 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-16T21:42:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be9a06640 1 Processor -- start 2023-12-16T21:42:42.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be9a06640 1 -- start start 2023-12-16T21:42:42.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be9a06640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 0x7f7be41403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be9a06640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40b6160 con 0x7f7be406fc70 2023-12-16T21:42:42.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be9a06640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be40b4590 con 0x7f7be40b4070 2023-12-16T21:42:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be9a06640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be4071100 con 0x7f7be40b5d80 2023-12-16T21:42:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7be406fc70 0x7f7be4071780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42974/0 (socket says 172.21.15.46:42974) 2023-12-16T21:42:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 -- 172.21.15.46:0/1421019844 learned_addr learned my addr 172.21.15.46:0/1421019844 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be2ffd640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 0x7f7be41403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2666774836 0 0) 0x7f7be40b6160 con 0x7f7be406fc70 2023-12-16T21:42:42.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bcc003530 con 0x7f7be406fc70 2023-12-16T21:42:42.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3948192707 0 0) 0x7f7bcc003530 con 0x7f7be406fc70 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 msgr2=0x7f7be41403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 0x7f7be41403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 >> v1:172.21.15.161:6789/0 conn(0x7f7be40b4070 legacy=0x7f7be4151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40b4870 con 0x7f7be406fc70 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f7bd4001680 con 0x7f7be406fc70 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 >> v1:172.21.15.46:6789/0 conn(0x7f7be406fc70 legacy=0x7f7be4071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bcc0039b0 0x7f7bcc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc0047d0 0x7f7bcc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 0x7f7bcc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be4071100 con 0x7f7bcc0047d0 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc00bac0 con 0x7f7bcc0039b0 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc00bc80 con 0x7f7be40b5d80 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc0047d0 0x7f7bcc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be2ffd640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bcc0039b0 0x7f7bcc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be37fe640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 0x7f7bcc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 -- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 msgr2=0x7f7bcc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7be40b5d80 0x7f7bcc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 -- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bcc0039b0 msgr2=0x7f7bcc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7bcc0039b0 0x7f7bcc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.191+0000 7f7be27fc640 1 -- 172.21.15.46:0/1421019844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc003530 con 0x7f7bcc0047d0 2023-12-16T21:42:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be27fc640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc0047d0 0x7f7bcc004c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400c210 tx=0x7f7bd401cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee51cccabf8a232 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:42.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd4005ae0 con 0x7f7bcc0047d0 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bd4005c40 con 0x7f7bcc0047d0 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be1ffb640 1 -- 172.21.15.46:0/1421019844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bd40a5860 con 0x7f7bcc0047d0 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- 172.21.15.46:0/1421019844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc0047d0 msgr2=0x7f7bcc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 --2- 172.21.15.46:0/1421019844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bcc0047d0 0x7f7bcc004c50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7bd400c210 tx=0x7f7bd401cbf0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be37fe640 1 -- 172.21.15.46:0/1421019844 reap_dead start 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- 172.21.15.46:0/1421019844 shutdown_connections 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- 172.21.15.46:0/1421019844 >> 172.21.15.46:0/1421019844 conn(0x7f7be406b860 msgr2=0x7f7be406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- 172.21.15.46:0/1421019844 shutdown_connections 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- 172.21.15.46:0/1421019844 wait complete. 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 Processor -- start 2023-12-16T21:42:42.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- start start 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bcc0047d0 0x7f7be41df190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be40ad010 0x7f7be41dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7be41e3360 0x7f7be4143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40ad600 con 0x7f7be41e3360 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be41e2ed0 con 0x7f7be40ad010 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.195+0000 7f7be9a06640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be40ae2c0 con 0x7f7bcc0047d0 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7be41e3360 0x7f7be4143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7be41e3360 0x7f7be4143610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44842/0 (socket says 172.21.15.46:44842) 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 -- 172.21.15.46:0/3778491707 learned_addr learned my addr 172.21.15.46:0/3778491707 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be27fc640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be40ad010 0x7f7be41dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be2ffd640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bcc0047d0 0x7f7be41df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 -- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bcc0047d0 msgr2=0x7f7be41df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bcc0047d0 0x7f7be41df190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 -- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be40ad010 msgr2=0x7f7be41dfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be40ad010 0x7f7be41dfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 -- 172.21.15.46:0/3778491707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be41e2880 con 0x7f7be41e3360 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be2ffd640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bcc0047d0 0x7f7be41df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be37fe640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7be41e3360 0x7f7be4143610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7be406c670 tx=0x7f7bdc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7bcbfff640 1 -- 172.21.15.46:0/3778491707 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc003980 con 0x7f7be41e3360 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7bcbfff640 1 -- 172.21.15.46:0/3778491707 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bdc003b00 con 0x7f7be41e3360 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be41e2b60 con 0x7f7be41e3360 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7bcbfff640 1 -- 172.21.15.46:0/3778491707 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc0050d0 con 0x7f7be41e3360 2023-12-16T21:42:42.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be40b1270 con 0x7f7be41e3360 2023-12-16T21:42:42.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be41e3060 con 0x7f7be41e3360 2023-12-16T21:42:42.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7bcbfff640 1 -- 172.21.15.46:0/3778491707 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7bdc005230 con 0x7f7be41e3360 2023-12-16T21:42:42.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7bcbfff640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7bd00664a0 0x7f7bd0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7bcbfff640 1 -- 172.21.15.46:0/3778491707 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bdc083c40 con 0x7f7be41e3360 2023-12-16T21:42:42.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be2ffd640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7bd00664a0 0x7f7bd0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.199+0000 7f7be2ffd640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7bd00664a0 0x7f7bd0068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009af0 tx=0x7f7bd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:42.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.203+0000 7f7bcbfff640 1 -- 172.21.15.46:0/3778491707 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7bdc051150 con 0x7f7be41e3360 2023-12-16T21:42:42.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.399+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7be41e3060 con 0x7f7be41e3360 2023-12-16T21:42:42.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.403+0000 7f7bcbfff640 1 -- 172.21.15.46:0/3778491707 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7bdc054770 con 0x7f7be41e3360 2023-12-16T21:42:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7bd00664a0 msgr2=0x7f7bd0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7bd00664a0 0x7f7bd0068960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7bd8009af0 tx=0x7f7bd8008040 comp rx=0 tx=0).stop 2023-12-16T21:42:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7be41e3360 msgr2=0x7f7be4143610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7be41e3360 0x7f7be4143610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7be406c670 tx=0x7f7bdc00ec10 comp rx=0 tx=0).stop 2023-12-16T21:42:42.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 shutdown_connections 2023-12-16T21:42:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7bd00664a0 0x7f7bd0068960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7be41e3360 0x7f7be4143610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7be40ad010 0x7f7be41dfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 --2- 172.21.15.46:0/3778491707 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7bcc0047d0 0x7f7be41df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 >> 172.21.15.46:0/3778491707 conn(0x7f7be406b860 msgr2=0x7f7be413e140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 shutdown_connections 2023-12-16T21:42:42.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.407+0000 7f7be9a06640 1 -- 172.21.15.46:0/3778491707 wait complete. 2023-12-16T21:42:42.432 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-16T21:42:42.432 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-16T21:42:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b95d2b640 1 Processor -- start 2023-12-16T21:42:42.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b95d2b640 1 -- start start 2023-12-16T21:42:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b95d2b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b9006c8b0 0x7f7b90149b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b95d2b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7b9014a050 0x7f7b90150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b95d2b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b900aee20 con 0x7f7b900721c0 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b95d2b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b900b4650 con 0x7f7b9006c8b0 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b95d2b640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9006cc90 con 0x7f7b9014a050 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b9006c8b0 0x7f7b90149b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7b9006c8b0 0x7f7b90149b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44858/0 (socket says 172.21.15.46:44858) 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 -- 172.21.15.46:0/360624851 learned_addr learned my addr 172.21.15.46:0/360624851 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8ffff640 1 --2- 172.21.15.46:0/360624851 >> v2:172.21.15.161:3300/0 conn(0x7f7b9014a050 0x7f7b90150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 -- 172.21.15.46:0/360624851 >> v2:172.21.15.161:3300/0 conn(0x7f7b9014a050 msgr2=0x7f7b90150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 --2- 172.21.15.46:0/360624851 >> v2:172.21.15.161:3300/0 conn(0x7f7b9014a050 0x7f7b90150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 -- 172.21.15.46:0/360624851 >> v1:172.21.15.46:6789/0 conn(0x7f7b900721c0 legacy=0x7f7b900b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 -- 172.21.15.46:0/360624851 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900b3b70 con 0x7f7b9006c8b0 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/360624851 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006302673 0 0) 0x7f7b900aee20 con 0x7f7b900721c0 2023-12-16T21:42:42.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8ffff640 1 --2- 172.21.15.46:0/360624851 >> v2:172.21.15.161:3300/0 conn(0x7f7b9014a050 0x7f7b90150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8f7fe640 1 --2- 172.21.15.46:0/360624851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9006c8b0 0x7f7b90149b10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7b90072bf0 tx=0x7f7b8001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65b92ff9ab163b35 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/360624851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800ad660 con 0x7f7b9006c8b0 2023-12-16T21:42:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/360624851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b800ad7c0 con 0x7f7b9006c8b0 2023-12-16T21:42:42.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.555+0000 7f7b8e7fc640 1 -- 172.21.15.46:0/360624851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b800adb50 con 0x7f7b9006c8b0 2023-12-16T21:42:42.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- 172.21.15.46:0/360624851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9006c8b0 msgr2=0x7f7b90149b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/360624851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9006c8b0 0x7f7b90149b10 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7b90072bf0 tx=0x7f7b8001c470 comp rx=0 tx=0).stop 2023-12-16T21:42:42.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- 172.21.15.46:0/360624851 shutdown_connections 2023-12-16T21:42:42.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/360624851 >> v2:172.21.15.161:3300/0 conn(0x7f7b9014a050 0x7f7b90150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/360624851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b9006c8b0 0x7f7b90149b10 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- 172.21.15.46:0/360624851 >> 172.21.15.46:0/360624851 conn(0x7f7b900b4900 msgr2=0x7f7b900b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:42.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- 172.21.15.46:0/360624851 shutdown_connections 2023-12-16T21:42:42.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- 172.21.15.46:0/360624851 wait complete. 2023-12-16T21:42:42.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 Processor -- start 2023-12-16T21:42:42.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- start start 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b9006c8b0 0x7f7b90142840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9014a050 0x7f7b9006f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b901435d0 0x7f7b9013f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90141b30 con 0x7f7b901435d0 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b90141830 con 0x7f7b9014a050 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901431c0 con 0x7f7b9006c8b0 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b901435d0 0x7f7b9013f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b901435d0 0x7f7b9013f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44874/0 (socket says 172.21.15.46:44874) 2023-12-16T21:42:42.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 -- 172.21.15.46:0/376534695 learned_addr learned my addr 172.21.15.46:0/376534695 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8f7fe640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b9006c8b0 0x7f7b90142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8effd640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9014a050 0x7f7b9006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 -- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b9006c8b0 msgr2=0x7f7b90142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b9006c8b0 0x7f7b90142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 -- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9014a050 msgr2=0x7f7b9006f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9014a050 0x7f7b9006f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 -- 172.21.15.46:0/376534695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b800094d0 con 0x7f7b901435d0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b8ffff640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b901435d0 0x7f7b9013f470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800cca0 tx=0x7f7b8800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b77fff640 1 -- 172.21.15.46:0/376534695 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800f680 con 0x7f7b901435d0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b77fff640 1 -- 172.21.15.46:0/376534695 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8800f800 con 0x7f7b901435d0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b90143f30 con 0x7f7b901435d0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b77fff640 1 -- 172.21.15.46:0/376534695 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800fbb0 con 0x7f7b901435d0 2023-12-16T21:42:42.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.559+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9006fe40 con 0x7f7b901435d0 2023-12-16T21:42:42.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.563+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b90141c70 con 0x7f7b901435d0 2023-12-16T21:42:42.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.563+0000 7f7b77fff640 1 -- 172.21.15.46:0/376534695 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7b8800b040 con 0x7f7b901435d0 2023-12-16T21:42:42.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.563+0000 7f7b77fff640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b7c066470 0x7f7b7c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:42.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.563+0000 7f7b77fff640 1 -- 172.21.15.46:0/376534695 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b88084750 con 0x7f7b901435d0 2023-12-16T21:42:42.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.563+0000 7f7b8f7fe640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b7c066470 0x7f7b7c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:42.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.563+0000 7f7b8f7fe640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b7c066470 0x7f7b7c068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7b800af000 tx=0x7f7b80008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:42.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.567+0000 7f7b77fff640 1 -- 172.21.15.46:0/376534695 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7b88051c60 con 0x7f7b901435d0 2023-12-16T21:42:42.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:42.755+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7b900ad090 con 0x7f7b901435d0 2023-12-16T21:42:43.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.431+0000 7f7b77fff640 1 -- 172.21.15.46:0/376534695 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7b88055280 con 0x7f7b901435d0 2023-12-16T21:42:43.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b7c066470 msgr2=0x7f7b7c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b7c066470 0x7f7b7c068930 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7b800af000 tx=0x7f7b80008040 comp rx=0 tx=0).stop 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b901435d0 msgr2=0x7f7b9013f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b901435d0 0x7f7b9013f470 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800cca0 tx=0x7f7b8800a650 comp rx=0 tx=0).stop 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 shutdown_connections 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b7c066470 0x7f7b7c068930 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7b901435d0 0x7f7b9013f470 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7b9014a050 0x7f7b9006f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 --2- 172.21.15.46:0/376534695 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7b9006c8b0 0x7f7b90142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 >> 172.21.15.46:0/376534695 conn(0x7f7b900b4900 msgr2=0x7f7b900b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 shutdown_connections 2023-12-16T21:42:43.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.435+0000 7f7b95d2b640 1 -- 172.21.15.46:0/376534695 wait complete. 2023-12-16T21:42:43.464 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-16T21:42:43.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:42:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.579+0000 7f80097f5640 1 Processor -- start 2023-12-16T21:42:43.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.579+0000 7f80097f5640 1 -- start start 2023-12-16T21:42:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80097f5640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f800406c920 0x7f80040afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80097f5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409dd50 0x7f800413ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80097f5640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80040af2e0 con 0x7f80040b3fd0 2023-12-16T21:42:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80097f5640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80040b1f50 con 0x7f800406c920 2023-12-16T21:42:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80097f5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800406cd00 con 0x7f800409dd50 2023-12-16T21:42:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80037fe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409dd50 0x7f800413ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8002ffd640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f800406c920 0x7f80040afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80037fe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409dd50 0x7f800413ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50306/0 (socket says 172.21.15.46:50306) 2023-12-16T21:42:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80037fe640 1 -- 172.21.15.46:0/3585484306 learned_addr learned my addr 172.21.15.46:0/3585484306 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80027fc640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f80040b3fd0 0x7f800413e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:54726/0 (socket says 172.21.15.46:54726) 2023-12-16T21:42:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8002ffd640 1 -- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409dd50 msgr2=0x7f800413ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8002ffd640 1 --2- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409dd50 0x7f800413ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8002ffd640 1 -- 172.21.15.46:0/3585484306 >> v1:172.21.15.161:6789/0 conn(0x7f80040b3fd0 legacy=0x7f800413e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8002ffd640 1 -- 172.21.15.46:0/3585484306 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80040af530 con 0x7f800406c920 2023-12-16T21:42:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f80037fe640 1 --2- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409dd50 0x7f800413ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8002ffd640 1 --2- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f80040afc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009d90 tx=0x7f7ff401c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1a17a7afa83934 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8001ffb640 1 -- 172.21.15.46:0/3585484306 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40a5780 con 0x7f800406c920 2023-12-16T21:42:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8001ffb640 1 -- 172.21.15.46:0/3585484306 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff40a58e0 con 0x7f800406c920 2023-12-16T21:42:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.583+0000 7f8001ffb640 1 -- 172.21.15.46:0/3585484306 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40a5c70 con 0x7f800406c920 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 msgr2=0x7f80040afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 --2- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f80040afc30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009d90 tx=0x7f7ff401c8c0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- 172.21.15.46:0/3585484306 shutdown_connections 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 --2- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409dd50 0x7f800413ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 --2- 172.21.15.46:0/3585484306 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f80040afc30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- 172.21.15.46:0/3585484306 >> 172.21.15.46:0/3585484306 conn(0x7f80040b48e0 msgr2=0x7f80040b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- 172.21.15.46:0/3585484306 shutdown_connections 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- 172.21.15.46:0/3585484306 wait complete. 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 Processor -- start 2023-12-16T21:42:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- start start 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f800413cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f800409dd50 0x7f800406e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409b130 0x7f800406ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040703b0 con 0x7f800409dd50 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040718d0 con 0x7f800406c920 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8004071300 con 0x7f800409b130 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f800409dd50 0x7f800406e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f800409dd50 0x7f800406e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44882/0 (socket says 172.21.15.46:44882) 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 -- 172.21.15.46:0/3611832555 learned_addr learned my addr 172.21.15.46:0/3611832555 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:43.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 -- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409b130 msgr2=0x7f800406ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409b130 0x7f800406ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 -- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 msgr2=0x7f800413cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f8002ffd640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f800413cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f800413cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 -- 172.21.15.46:0/3611832555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc0094d0 con 0x7f800409dd50 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f8002ffd640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f800413cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80027fc640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f800409dd50 0x7f800406e8d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c370 tx=0x7f7ff800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f7feb7fe640 1 -- 172.21.15.46:0/3611832555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff80074e0 con 0x7f800409dd50 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f7feb7fe640 1 -- 172.21.15.46:0/3611832555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff8007640 con 0x7f800409dd50 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff4009a70 con 0x7f800409dd50 2023-12-16T21:42:43.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f7feb7fe640 1 -- 172.21.15.46:0/3611832555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8004a30 con 0x7f800409dd50 2023-12-16T21:42:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.587+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80040714f0 con 0x7f800409dd50 2023-12-16T21:42:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.591+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80040679e0 con 0x7f800409dd50 2023-12-16T21:42:43.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.591+0000 7f7feb7fe640 1 -- 172.21.15.46:0/3611832555 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7ff8004b90 con 0x7f800409dd50 2023-12-16T21:42:43.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.591+0000 7f7feb7fe640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7ff0066470 0x7f7ff0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.591+0000 7f7feb7fe640 1 -- 172.21.15.46:0/3611832555 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ff80837a0 con 0x7f800409dd50 2023-12-16T21:42:43.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.591+0000 7f8002ffd640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7ff0066470 0x7f7ff0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.591+0000 7f8002ffd640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7ff0066470 0x7f7ff0068930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40af000 tx=0x7f7ff40a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:43.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.595+0000 7f7feb7fe640 1 -- 172.21.15.46:0/3611832555 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7ff8050cb0 con 0x7f800409dd50 2023-12-16T21:42:43.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8004070210 con 0x7f800409dd50 2023-12-16T21:42:43.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f7feb7fe640 1 -- 172.21.15.46:0/3611832555 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7f8004070210 con 0x7f800409dd50 2023-12-16T21:42:43.786 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:43.787 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:42:43.787 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:42:43.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7ff0066470 msgr2=0x7f7ff0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7ff0066470 0x7f7ff0068930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40af000 tx=0x7f7ff40a7070 comp rx=0 tx=0).stop 2023-12-16T21:42:43.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f800409dd50 msgr2=0x7f800406e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f800409dd50 0x7f800406e8d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800c370 tx=0x7f7ff800c830 comp rx=0 tx=0).stop 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 shutdown_connections 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7ff0066470 0x7f7ff0068930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f800409b130 0x7f800406ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f800409dd50 0x7f800406e8d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 --2- 172.21.15.46:0/3611832555 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f800406c920 0x7f800413cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 >> 172.21.15.46:0/3611832555 conn(0x7f80040b48e0 msgr2=0x7f80040ae9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 shutdown_connections 2023-12-16T21:42:43.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.779+0000 7f80097f5640 1 -- 172.21.15.46:0/3611832555 wait complete. 2023-12-16T21:42:43.804 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f99683b7640 1 Processor -- start 2023-12-16T21:42:43.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f99683b7640 1 -- start start 2023-12-16T21:42:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f99683b7640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f99600b4520 0x7f9960144690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f99683b7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9960071fd0 0x7f9960144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f99683b7640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99600b1120 con 0x7f99600b5150 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f99683b7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99600b04a0 con 0x7f9960071fd0 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f99683b7640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99600b3a10 con 0x7f99600b4520 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9966954640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9960071fd0 0x7f9960144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9966954640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f9960071fd0 0x7f9960144bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44894/0 (socket says 172.21.15.46:44894) 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9966954640 1 -- 172.21.15.46:0/1982390000 learned_addr learned my addr 172.21.15.46:0/1982390000 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9965952640 1 --2- 172.21.15.46:0/1982390000 >> v2:172.21.15.161:3300/0 conn(0x7f99600b4520 0x7f9960144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9966954640 1 -- 172.21.15.46:0/1982390000 >> v2:172.21.15.161:3300/0 conn(0x7f99600b4520 msgr2=0x7f9960144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9966954640 1 --2- 172.21.15.46:0/1982390000 >> v2:172.21.15.161:3300/0 conn(0x7f99600b4520 0x7f9960144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9966954640 1 -- 172.21.15.46:0/1982390000 >> v1:172.21.15.161:6789/0 conn(0x7f99600b5150 legacy=0x7f99600b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.919+0000 7f9966954640 1 -- 172.21.15.46:0/1982390000 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600b3cf0 con 0x7f9960071fd0 2023-12-16T21:42:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9966954640 1 --2- 172.21.15.46:0/1982390000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9960071fd0 0x7f9960144bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99600b05e0 tx=0x7f995801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7990ffc856d1def server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:43.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965151640 1 -- 172.21.15.46:0/1982390000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99580a5790 con 0x7f9960071fd0 2023-12-16T21:42:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965151640 1 -- 172.21.15.46:0/1982390000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99580a58f0 con 0x7f9960071fd0 2023-12-16T21:42:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965151640 1 -- 172.21.15.46:0/1982390000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99580a5c80 con 0x7f9960071fd0 2023-12-16T21:42:43.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- 172.21.15.46:0/1982390000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9960071fd0 msgr2=0x7f9960144bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 --2- 172.21.15.46:0/1982390000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9960071fd0 0x7f9960144bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99600b05e0 tx=0x7f995801c8b0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- 172.21.15.46:0/1982390000 shutdown_connections 2023-12-16T21:42:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 --2- 172.21.15.46:0/1982390000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9960071fd0 0x7f9960144bd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 --2- 172.21.15.46:0/1982390000 >> v2:172.21.15.161:3300/0 conn(0x7f99600b4520 0x7f9960144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- 172.21.15.46:0/1982390000 >> 172.21.15.46:0/1982390000 conn(0x7f99600b9250 msgr2=0x7f99600b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- 172.21.15.46:0/1982390000 shutdown_connections 2023-12-16T21:42:43.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- 172.21.15.46:0/1982390000 wait complete. 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 Processor -- start 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- start start 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9960071fd0 0x7f996013dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99600b4520 0x7f99601d69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99600b5150 0x7f99601d6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99601ddef0 con 0x7f99600b4520 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996013e950 con 0x7f99600b5150 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99601dd5b0 con 0x7f9960071fd0 2023-12-16T21:42:43.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99600b4520 0x7f99601d69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99600b4520 0x7f99601d69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44902/0 (socket says 172.21.15.46:44902) 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 -- 172.21.15.46:0/284713366 learned_addr learned my addr 172.21.15.46:0/284713366 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9966954640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99600b5150 0x7f99601d6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9966153640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9960071fd0 0x7f996013dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 -- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9960071fd0 msgr2=0x7f996013dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9960071fd0 0x7f996013dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 -- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99600b5150 msgr2=0x7f99601d6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99600b5150 0x7f99601d6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 -- 172.21.15.46:0/284713366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9958009b30 con 0x7f99600b4520 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9965952640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99600b4520 0x7f99601d69d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f995000e960 tx=0x7f995000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:43.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9956ffd640 1 -- 172.21.15.46:0/284713366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950009690 con 0x7f99600b4520 2023-12-16T21:42:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9956ffd640 1 -- 172.21.15.46:0/284713366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000ca00 con 0x7f99600b4520 2023-12-16T21:42:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99601dd830 con 0x7f99600b4520 2023-12-16T21:42:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f9956ffd640 1 -- 172.21.15.46:0/284713366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995000cd70 con 0x7f99600b4520 2023-12-16T21:42:43.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.923+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996013b2d0 con 0x7f99600b4520 2023-12-16T21:42:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.927+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99601de030 con 0x7f99600b4520 2023-12-16T21:42:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.927+0000 7f9956ffd640 1 -- 172.21.15.46:0/284713366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f9950013470 con 0x7f99600b4520 2023-12-16T21:42:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.927+0000 7f9956ffd640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9944066470 0x7f9944068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.927+0000 7f9956ffd640 1 -- 172.21.15.46:0/284713366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9950084030 con 0x7f99600b4520 2023-12-16T21:42:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.927+0000 7f9966153640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9944066470 0x7f9944068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:43.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.927+0000 7f9966153640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9944066470 0x7f9944068930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9960138e90 tx=0x7f995c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:43.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:43.931+0000 7f9956ffd640 1 -- 172.21.15.46:0/284713366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f9950051540 con 0x7f99600b4520 2023-12-16T21:42:44.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.083+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99600688f0 con 0x7f99600b4520 2023-12-16T21:42:44.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.083+0000 7f9956ffd640 1 -- 172.21.15.46:0/284713366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f9950054b60 con 0x7f99600b4520 2023-12-16T21:42:44.092 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:44.092 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9944066470 msgr2=0x7f9944068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9944066470 0x7f9944068930 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9960138e90 tx=0x7f995c009040 comp rx=0 tx=0).stop 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99600b4520 msgr2=0x7f99601d69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99600b4520 0x7f99601d69d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f995000e960 tx=0x7f995000ee20 comp rx=0 tx=0).stop 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 shutdown_connections 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9944066470 0x7f9944068930 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f99600b5150 0x7f99601d6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99600b4520 0x7f99601d69d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 --2- 172.21.15.46:0/284713366 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9960071fd0 0x7f996013dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 >> 172.21.15.46:0/284713366 conn(0x7f99600b9250 msgr2=0x7f99600b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 shutdown_connections 2023-12-16T21:42:44.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.087+0000 7f99683b7640 1 -- 172.21.15.46:0/284713366 wait complete. 2023-12-16T21:42:44.109 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-16T21:42:44.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:42:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48bd06640 1 Processor -- start 2023-12-16T21:42:44.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48bd06640 1 -- start start 2023-12-16T21:42:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48bd06640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb4840b3fd0 0x7fb48413e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48bd06640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb4840b0340 0x7fb48413e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48bd06640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4840af2e0 con 0x7fb48406c920 2023-12-16T21:42:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48bd06640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4840b1f50 con 0x7fb4840b3fd0 2023-12-16T21:42:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48bd06640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48413eda0 con 0x7fb4840b0340 2023-12-16T21:42:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb4840b3fd0 0x7fb48413e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb4840b3fd0 0x7fb48413e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44904/0 (socket says 172.21.15.46:44904) 2023-12-16T21:42:44.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 -- 172.21.15.46:0/3227957747 learned_addr learned my addr 172.21.15.46:0/3227957747 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:44.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb48a2a3640 1 --2- 172.21.15.46:0/3227957747 >> v2:172.21.15.161:3300/0 conn(0x7fb4840b0340 0x7fb48413e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 -- 172.21.15.46:0/3227957747 >> v2:172.21.15.161:3300/0 conn(0x7fb4840b0340 msgr2=0x7fb48413e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:44.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 --2- 172.21.15.46:0/3227957747 >> v2:172.21.15.161:3300/0 conn(0x7fb4840b0340 0x7fb48413e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 -- 172.21.15.46:0/3227957747 >> v1:172.21.15.161:6789/0 conn(0x7fb48406c920 legacy=0x7fb4840afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:44.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 -- 172.21.15.46:0/3227957747 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48406cd00 con 0x7fb4840b3fd0 2023-12-16T21:42:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb4892a1640 1 --2- 172.21.15.46:0/3227957747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840b3fd0 0x7fb48413e320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4840b0f90 tx=0x7fb47401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedf27d81de8c32f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb488aa0640 1 -- 172.21.15.46:0/3227957747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740ad680 con 0x7fb4840b3fd0 2023-12-16T21:42:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb488aa0640 1 -- 172.21.15.46:0/3227957747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740ad7e0 con 0x7fb4840b3fd0 2023-12-16T21:42:44.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.211+0000 7fb488aa0640 1 -- 172.21.15.46:0/3227957747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740adb70 con 0x7fb4840b3fd0 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- 172.21.15.46:0/3227957747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840b3fd0 msgr2=0x7fb48413e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 --2- 172.21.15.46:0/3227957747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840b3fd0 0x7fb48413e320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4840b0f90 tx=0x7fb47401c470 comp rx=0 tx=0).stop 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- 172.21.15.46:0/3227957747 shutdown_connections 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 --2- 172.21.15.46:0/3227957747 >> v2:172.21.15.161:3300/0 conn(0x7fb4840b0340 0x7fb48413e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 --2- 172.21.15.46:0/3227957747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4840b3fd0 0x7fb48413e320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- 172.21.15.46:0/3227957747 >> 172.21.15.46:0/3227957747 conn(0x7fb4840b48e0 msgr2=0x7fb4840b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- 172.21.15.46:0/3227957747 shutdown_connections 2023-12-16T21:42:44.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- 172.21.15.46:0/3227957747 wait complete. 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 Processor -- start 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- start start 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb48406c920 0x7fb48413d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4841dd470 0x7fb48413f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4841dd8f0 0x7fb484133e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48413ef60 con 0x7fb48406c920 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841dcc90 con 0x7fb4841dd470 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4841401c0 con 0x7fb4841dd8f0 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb48406c920 0x7fb48413d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb48406c920 0x7fb48413d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44912/0 (socket says 172.21.15.46:44912) 2023-12-16T21:42:44.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 -- 172.21.15.46:0/2929604490 learned_addr learned my addr 172.21.15.46:0/2929604490 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb4892a1640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4841dd470 0x7fb48413f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48a2a3640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4841dd8f0 0x7fb484133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 -- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4841dd8f0 msgr2=0x7fb484133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4841dd8f0 0x7fb484133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 -- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4841dd470 msgr2=0x7fb48413f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4841dd470 0x7fb48413f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 -- 172.21.15.46:0/2929604490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4740094d0 con 0x7fb48406c920 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb4892a1640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4841dd470 0x7fb48413f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb489aa2640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb48406c920 0x7fb48413d430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c500 tx=0x7fb48000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb47a7fc640 1 -- 172.21.15.46:0/2929604490 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480005220 con 0x7fb48406c920 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb47a7fc640 1 -- 172.21.15.46:0/2929604490 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480005380 con 0x7fb48406c920 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb47a7fc640 1 -- 172.21.15.46:0/2929604490 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480007270 con 0x7fb48406c920 2023-12-16T21:42:44.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48413f200 con 0x7fb48406c920 2023-12-16T21:42:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.215+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb484140930 con 0x7fb48406c920 2023-12-16T21:42:44.226 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.219+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4840679e0 con 0x7fb48406c920 2023-12-16T21:42:44.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.219+0000 7fb47a7fc640 1 -- 172.21.15.46:0/2929604490 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fb4800073d0 con 0x7fb48406c920 2023-12-16T21:42:44.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.219+0000 7fb47a7fc640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb468066470 0x7fb468068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.219+0000 7fb47a7fc640 1 -- 172.21.15.46:0/2929604490 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb480083760 con 0x7fb48406c920 2023-12-16T21:42:44.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.219+0000 7fb4892a1640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb468066470 0x7fb468068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.219+0000 7fb4892a1640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb468066470 0x7fb468068930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4740af000 tx=0x7fb474008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.223+0000 7fb47a7fc640 1 -- 172.21.15.46:0/2929604490 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fb480050bf0 con 0x7fb48406c920 2023-12-16T21:42:44.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.371+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4841dcdd0 con 0x7fb468066470 2023-12-16T21:42:44.398 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.391+0000 7fb47a7fc640 1 -- 172.21.15.46:0/2929604490 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264563 (secure 0 0 0) 0x7fb4740bc090 con 0x7fb468066470 2023-12-16T21:42:44.401 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:44.403 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb468066470 msgr2=0x7fb468068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb468066470 0x7fb468068930 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb4740af000 tx=0x7fb474008040 comp rx=0 tx=0).stop 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb48406c920 msgr2=0x7fb48413d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb48406c920 0x7fb48413d430 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb48000c500 tx=0x7fb48000c9c0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 shutdown_connections 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb468066470 0x7fb468068930 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb4841dd8f0 0x7fb484133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb4841dd470 0x7fb48413f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 --2- 172.21.15.46:0/2929604490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb48406c920 0x7fb48413d430 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 >> 172.21.15.46:0/2929604490 conn(0x7fb4840b48e0 msgr2=0x7fb4840b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 shutdown_connections 2023-12-16T21:42:44.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.395+0000 7fb48bd06640 1 -- 172.21.15.46:0/2929604490 wait complete. 2023-12-16T21:42:44.417 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-16T21:42:44.351441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235660,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739060,"statfs":{"total":773094113280,"available":772852797440,"internally_reserved":0,"allocated":2891776,"data_stored":1739804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.963078"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333636+0000","last_change":"2023-12-16T21:42:41.368574+0000","last_active":"2023-12-16T21:42:42.333636+0000","last_peered":"2023-12-16T21:42:42.333636+0000","last_clean":"2023-12-16T21:42:42.333636+0000","last_became_active":"2023-12-16T21:42:41.367555+0000","last_became_peered":"2023-12-16T21:42:41.367555+0000","last_unstale":"2023-12-16T21:42:42.333636+0000","last_undegraded":"2023-12-16T21:42:42.333636+0000","last_fullsized":"2023-12-16T21:42:42.333636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329805+0000","last_change":"2023-12-16T21:42:41.344487+0000","last_active":"2023-12-16T21:42:42.329805+0000","last_peered":"2023-12-16T21:42:42.329805+0000","last_clean":"2023-12-16T21:42:42.329805+0000","last_became_active":"2023-12-16T21:42:41.343798+0000","last_became_peered":"2023-12-16T21:42:41.343798+0000","last_unstale":"2023-12-16T21:42:42.329805+0000","last_undegraded":"2023-12-16T21:42:42.329805+0000","last_fullsized":"2023-12-16T21:42:42.329805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471802+0000","last_change":"2023-12-16T21:42:41.364221+0000","last_active":"2023-12-16T21:42:42.471802+0000","last_peered":"2023-12-16T21:42:42.471802+0000","last_clean":"2023-12-16T21:42:42.471802+0000","last_became_active":"2023-12-16T21:42:41.362506+0000","last_became_peered":"2023-12-16T21:42:41.362506+0000","last_unstale":"2023-12-16T21:42:42.471802+0000","last_undegraded":"2023-12-16T21:42:42.471802+0000","last_fullsized":"2023-12-16T21:42:42.471802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474631+0000","last_change":"2023-12-16T21:42:41.384692+0000","last_active":"2023-12-16T21:42:42.474631+0000","last_peered":"2023-12-16T21:42:42.474631+0000","last_clean":"2023-12-16T21:42:42.474631+0000","last_became_active":"2023-12-16T21:42:41.384156+0000","last_became_peered":"2023-12-16T21:42:41.384156+0000","last_unstale":"2023-12-16T21:42:42.474631+0000","last_undegraded":"2023-12-16T21:42:42.474631+0000","last_fullsized":"2023-12-16T21:42:42.474631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333576+0000","last_change":"2023-12-16T21:42:41.355491+0000","last_active":"2023-12-16T21:42:42.333576+0000","last_peered":"2023-12-16T21:42:42.333576+0000","last_clean":"2023-12-16T21:42:42.333576+0000","last_became_active":"2023-12-16T21:42:41.354519+0000","last_became_peered":"2023-12-16T21:42:41.354519+0000","last_unstale":"2023-12-16T21:42:42.333576+0000","last_undegraded":"2023-12-16T21:42:42.333576+0000","last_fullsized":"2023-12-16T21:42:42.333576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474628+0000","last_change":"2023-12-16T21:42:41.367454+0000","last_active":"2023-12-16T21:42:42.474628+0000","last_peered":"2023-12-16T21:42:42.474628+0000","last_clean":"2023-12-16T21:42:42.474628+0000","last_became_active":"2023-12-16T21:42:41.366663+0000","last_became_peered":"2023-12-16T21:42:41.366663+0000","last_unstale":"2023-12-16T21:42:42.474628+0000","last_undegraded":"2023-12-16T21:42:42.474628+0000","last_fullsized":"2023-12-16T21:42:42.474628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332459+0000","last_change":"2023-12-16T21:42:41.375370+0000","last_active":"2023-12-16T21:42:42.332459+0000","last_peered":"2023-12-16T21:42:42.332459+0000","last_clean":"2023-12-16T21:42:42.332459+0000","last_became_active":"2023-12-16T21:42:41.374733+0000","last_became_peered":"2023-12-16T21:42:41.374733+0000","last_unstale":"2023-12-16T21:42:42.332459+0000","last_undegraded":"2023-12-16T21:42:42.332459+0000","last_fullsized":"2023-12-16T21:42:42.332459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471989+0000","last_change":"2023-12-16T21:42:41.385909+0000","last_active":"2023-12-16T21:42:42.471989+0000","last_peered":"2023-12-16T21:42:42.471989+0000","last_clean":"2023-12-16T21:42:42.471989+0000","last_became_active":"2023-12-16T21:42:41.385386+0000","last_became_peered":"2023-12-16T21:42:41.385386+0000","last_unstale":"2023-12-16T21:42:42.471989+0000","last_undegraded":"2023-12-16T21:42:42.471989+0000","last_fullsized":"2023-12-16T21:42:42.471989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333967+0000","last_change":"2023-12-16T21:42:41.374260+0000","last_active":"2023-12-16T21:42:42.333967+0000","last_peered":"2023-12-16T21:42:42.333967+0000","last_clean":"2023-12-16T21:42:42.333967+0000","last_became_active":"2023-12-16T21:42:41.373101+0000","last_became_peered":"2023-12-16T21:42:41.373101+0000","last_unstale":"2023-12-16T21:42:42.333967+0000","last_undegraded":"2023-12-16T21:42:42.333967+0000","last_fullsized":"2023-12-16T21:42:42.333967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475842+0000","last_change":"2023-12-16T21:42:41.386312+0000","last_active":"2023-12-16T21:42:42.475842+0000","last_peered":"2023-12-16T21:42:42.475842+0000","last_clean":"2023-12-16T21:42:42.475842+0000","last_became_active":"2023-12-16T21:42:41.385651+0000","last_became_peered":"2023-12-16T21:42:41.385651+0000","last_unstale":"2023-12-16T21:42:42.475842+0000","last_undegraded":"2023-12-16T21:42:42.475842+0000","last_fullsized":"2023-12-16T21:42:42.475842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475260+0000","last_change":"2023-12-16T21:42:41.385382+0000","last_active":"2023-12-16T21:42:42.475260+0000","last_peered":"2023-12-16T21:42:42.475260+0000","last_clean":"2023-12-16T21:42:42.475260+0000","last_became_active":"2023-12-16T21:42:41.384671+0000","last_became_peered":"2023-12-16T21:42:41.384671+0000","last_unstale":"2023-12-16T21:42:42.475260+0000","last_undegraded":"2023-12-16T21:42:42.475260+0000","last_fullsized":"2023-12-16T21:42:42.475260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335258+0000","last_change":"2023-12-16T21:42:41.373439+0000","last_active":"2023-12-16T21:42:42.335258+0000","last_peered":"2023-12-16T21:42:42.335258+0000","last_clean":"2023-12-16T21:42:42.335258+0000","last_became_active":"2023-12-16T21:42:41.372462+0000","last_became_peered":"2023-12-16T21:42:41.372462+0000","last_unstale":"2023-12-16T21:42:42.335258+0000","last_undegraded":"2023-12-16T21:42:42.335258+0000","last_fullsized":"2023-12-16T21:42:42.335258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471349+0000","last_change":"2023-12-16T21:42:41.366622+0000","last_active":"2023-12-16T21:42:42.471349+0000","last_peered":"2023-12-16T21:42:42.471349+0000","last_clean":"2023-12-16T21:42:42.471349+0000","last_became_active":"2023-12-16T21:42:41.365732+0000","last_became_peered":"2023-12-16T21:42:41.365732+0000","last_unstale":"2023-12-16T21:42:42.471349+0000","last_undegraded":"2023-12-16T21:42:42.471349+0000","last_fullsized":"2023-12-16T21:42:42.471349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472312+0000","last_change":"2023-12-16T21:42:41.347792+0000","last_active":"2023-12-16T21:42:42.472312+0000","last_peered":"2023-12-16T21:42:42.472312+0000","last_clean":"2023-12-16T21:42:42.472312+0000","last_became_active":"2023-12-16T21:42:41.345588+0000","last_became_peered":"2023-12-16T21:42:41.345588+0000","last_unstale":"2023-12-16T21:42:42.472312+0000","last_undegraded":"2023-12-16T21:42:42.472312+0000","last_fullsized":"2023-12-16T21:42:42.472312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329364+0000","last_change":"2023-12-16T21:42:41.343823+0000","last_active":"2023-12-16T21:42:42.329364+0000","last_peered":"2023-12-16T21:42:42.329364+0000","last_clean":"2023-12-16T21:42:42.329364+0000","last_became_active":"2023-12-16T21:42:41.343022+0000","last_became_peered":"2023-12-16T21:42:41.343022+0000","last_unstale":"2023-12-16T21:42:42.329364+0000","last_undegraded":"2023-12-16T21:42:42.329364+0000","last_fullsized":"2023-12-16T21:42:42.329364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474121+0000","last_change":"2023-12-16T21:42:41.383653+0000","last_active":"2023-12-16T21:42:42.474121+0000","last_peered":"2023-12-16T21:42:42.474121+0000","last_clean":"2023-12-16T21:42:42.474121+0000","last_became_active":"2023-12-16T21:42:41.383223+0000","last_became_peered":"2023-12-16T21:42:41.383223+0000","last_unstale":"2023-12-16T21:42:42.474121+0000","last_undegraded":"2023-12-16T21:42:42.474121+0000","last_fullsized":"2023-12-16T21:42:42.474121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476620+0000","last_change":"2023-12-16T21:42:41.366575+0000","last_active":"2023-12-16T21:42:42.476620+0000","last_peered":"2023-12-16T21:42:42.476620+0000","last_clean":"2023-12-16T21:42:42.476620+0000","last_became_active":"2023-12-16T21:42:41.365758+0000","last_became_peered":"2023-12-16T21:42:41.365758+0000","last_unstale":"2023-12-16T21:42:42.476620+0000","last_undegraded":"2023-12-16T21:42:42.476620+0000","last_fullsized":"2023-12-16T21:42:42.476620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471210+0000","last_change":"2023-12-16T21:42:41.371042+0000","last_active":"2023-12-16T21:42:42.471210+0000","last_peered":"2023-12-16T21:42:42.471210+0000","last_clean":"2023-12-16T21:42:42.471210+0000","last_became_active":"2023-12-16T21:42:41.370204+0000","last_became_peered":"2023-12-16T21:42:41.370204+0000","last_unstale":"2023-12-16T21:42:42.471210+0000","last_undegraded":"2023-12-16T21:42:42.471210+0000","last_fullsized":"2023-12-16T21:42:42.471210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335291+0000","last_change":"2023-12-16T21:42:41.360173+0000","last_active":"2023-12-16T21:42:42.335291+0000","last_peered":"2023-12-16T21:42:42.335291+0000","last_clean":"2023-12-16T21:42:42.335291+0000","last_became_active":"2023-12-16T21:42:41.359373+0000","last_became_peered":"2023-12-16T21:42:41.359373+0000","last_unstale":"2023-12-16T21:42:42.335291+0000","last_undegraded":"2023-12-16T21:42:42.335291+0000","last_fullsized":"2023-12-16T21:42:42.335291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:59:44.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475288+0000","last_change":"2023-12-16T21:42:41.365867+0000","last_active":"2023-12-16T21:42:42.475288+0000","last_peered":"2023-12-16T21:42:42.475288+0000","last_clean":"2023-12-16T21:42:42.475288+0000","last_became_active":"2023-12-16T21:42:41.365202+0000","last_became_peered":"2023-12-16T21:42:41.365202+0000","last_unstale":"2023-12-16T21:42:42.475288+0000","last_undegraded":"2023-12-16T21:42:42.475288+0000","last_fullsized":"2023-12-16T21:42:42.475288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471522+0000","last_change":"2023-12-16T21:42:41.377468+0000","last_active":"2023-12-16T21:42:42.471522+0000","last_peered":"2023-12-16T21:42:42.471522+0000","last_clean":"2023-12-16T21:42:42.471522+0000","last_became_active":"2023-12-16T21:42:41.376856+0000","last_became_peered":"2023-12-16T21:42:41.376856+0000","last_unstale":"2023-12-16T21:42:42.471522+0000","last_undegraded":"2023-12-16T21:42:42.471522+0000","last_fullsized":"2023-12-16T21:42:42.471522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:55.715961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475667+0000","last_change":"2023-12-16T21:42:41.366088+0000","last_active":"2023-12-16T21:42:42.475667+0000","last_peered":"2023-12-16T21:42:42.475667+0000","last_clean":"2023-12-16T21:42:42.475667+0000","last_became_active":"2023-12-16T21:42:41.365380+0000","last_became_peered":"2023-12-16T21:42:41.365380+0000","last_unstale":"2023-12-16T21:42:42.475667+0000","last_undegraded":"2023-12-16T21:42:42.475667+0000","last_fullsized":"2023-12-16T21:42:42.475667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471681+0000","last_change":"2023-12-16T21:42:41.378284+0000","last_active":"2023-12-16T21:42:42.471681+0000","last_peered":"2023-12-16T21:42:42.471681+0000","last_clean":"2023-12-16T21:42:42.471681+0000","last_became_active":"2023-12-16T21:42:41.377070+0000","last_became_peered":"2023-12-16T21:42:41.377070+0000","last_unstale":"2023-12-16T21:42:42.471681+0000","last_undegraded":"2023-12-16T21:42:42.471681+0000","last_fullsized":"2023-12-16T21:42:42.471681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475272+0000","last_change":"2023-12-16T21:42:41.376780+0000","last_active":"2023-12-16T21:42:42.475272+0000","last_peered":"2023-12-16T21:42:42.475272+0000","last_clean":"2023-12-16T21:42:42.475272+0000","last_became_active":"2023-12-16T21:42:41.376224+0000","last_became_peered":"2023-12-16T21:42:41.376224+0000","last_unstale":"2023-12-16T21:42:42.475272+0000","last_undegraded":"2023-12-16T21:42:42.475272+0000","last_fullsized":"2023-12-16T21:42:42.475272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472816+0000","last_change":"2023-12-16T21:42:41.372281+0000","last_active":"2023-12-16T21:42:42.472816+0000","last_peered":"2023-12-16T21:42:42.472816+0000","last_clean":"2023-12-16T21:42:42.472816+0000","last_became_active":"2023-12-16T21:42:41.371551+0000","last_became_peered":"2023-12-16T21:42:41.371551+0000","last_unstale":"2023-12-16T21:42:42.472816+0000","last_undegraded":"2023-12-16T21:42:42.472816+0000","last_fullsized":"2023-12-16T21:42:42.472816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475958+0000","last_change":"2023-12-16T21:42:41.375672+0000","last_active":"2023-12-16T21:42:42.475958+0000","last_peered":"2023-12-16T21:42:42.475958+0000","last_clean":"2023-12-16T21:42:42.475958+0000","last_became_active":"2023-12-16T21:42:41.370826+0000","last_became_peered":"2023-12-16T21:42:41.370826+0000","last_unstale":"2023-12-16T21:42:42.475958+0000","last_undegraded":"2023-12-16T21:42:42.475958+0000","last_fullsized":"2023-12-16T21:42:42.475958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329872+0000","last_change":"2023-12-16T21:42:41.368001+0000","last_active":"2023-12-16T21:42:42.329872+0000","last_peered":"2023-12-16T21:42:42.329872+0000","last_clean":"2023-12-16T21:42:42.329872+0000","last_became_active":"2023-12-16T21:42:41.367168+0000","last_became_peered":"2023-12-16T21:42:41.367168+0000","last_unstale":"2023-12-16T21:42:42.329872+0000","last_undegraded":"2023-12-16T21:42:42.329872+0000","last_fullsized":"2023-12-16T21:42:42.329872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472910+0000","last_change":"2023-12-16T21:42:41.377845+0000","last_active":"2023-12-16T21:42:42.472910+0000","last_peered":"2023-12-16T21:42:42.472910+0000","last_clean":"2023-12-16T21:42:42.472910+0000","last_became_active":"2023-12-16T21:42:41.377023+0000","last_became_peered":"2023-12-16T21:42:41.377023+0000","last_unstale":"2023-12-16T21:42:42.472910+0000","last_undegraded":"2023-12-16T21:42:42.472910+0000","last_fullsized":"2023-12-16T21:42:42.472910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472206+0000","last_change":"2023-12-16T21:42:41.371637+0000","last_active":"2023-12-16T21:42:42.472206+0000","last_peered":"2023-12-16T21:42:42.472206+0000","last_clean":"2023-12-16T21:42:42.472206+0000","last_became_active":"2023-12-16T21:42:41.370853+0000","last_became_peered":"2023-12-16T21:42:41.370853+0000","last_unstale":"2023-12-16T21:42:42.472206+0000","last_undegraded":"2023-12-16T21:42:42.472206+0000","last_fullsized":"2023-12-16T21:42:42.472206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475286+0000","last_change":"2023-12-16T21:42:41.375235+0000","last_active":"2023-12-16T21:42:42.475286+0000","last_peered":"2023-12-16T21:42:42.475286+0000","last_clean":"2023-12-16T21:42:42.475286+0000","last_became_active":"2023-12-16T21:42:41.374570+0000","last_became_peered":"2023-12-16T21:42:41.374570+0000","last_unstale":"2023-12-16T21:42:42.475286+0000","last_undegraded":"2023-12-16T21:42:42.475286+0000","last_fullsized":"2023-12-16T21:42:42.475286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472367+0000","last_change":"2023-12-16T21:42:41.383749+0000","last_active":"2023-12-16T21:42:42.472367+0000","last_peered":"2023-12-16T21:42:42.472367+0000","last_clean":"2023-12-16T21:42:42.472367+0000","last_became_active":"2023-12-16T21:42:41.383009+0000","last_became_peered":"2023-12-16T21:42:41.383009+0000","last_unstale":"2023-12-16T21:42:42.472367+0000","last_undegraded":"2023-12-16T21:42:42.472367+0000","last_fullsized":"2023-12-16T21:42:42.472367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473979+0000","last_change":"2023-12-16T21:42:41.379801+0000","last_active":"2023-12-16T21:42:42.473979+0000","last_peered":"2023-12-16T21:42:42.473979+0000","last_clean":"2023-12-16T21:42:42.473979+0000","last_became_active":"2023-12-16T21:42:41.379000+0000","last_became_peered":"2023-12-16T21:42:41.379000+0000","last_unstale":"2023-12-16T21:42:42.473979+0000","last_undegraded":"2023-12-16T21:42:42.473979+0000","last_fullsized":"2023-12-16T21:42:42.473979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333784+0000","last_change":"2023-12-16T21:42:41.351207+0000","last_active":"2023-12-16T21:42:42.333784+0000","last_peered":"2023-12-16T21:42:42.333784+0000","last_clean":"2023-12-16T21:42:42.333784+0000","last_became_active":"2023-12-16T21:42:41.349156+0000","last_became_peered":"2023-12-16T21:42:41.349156+0000","last_unstale":"2023-12-16T21:42:42.333784+0000","last_undegraded":"2023-12-16T21:42:42.333784+0000","last_fullsized":"2023-12-16T21:42:42.333784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:52.130538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472650+0000","last_change":"2023-12-16T21:42:40.442950+0000","last_active":"2023-12-16T21:42:42.472650+0000","last_peered":"2023-12-16T21:42:42.472650+0000","last_clean":"2023-12-16T21:42:42.472650+0000","last_became_active":"2023-12-16T21:42:40.442532+0000","last_became_peered":"2023-12-16T21:42:40.442532+0000","last_unstale":"2023-12-16T21:42:42.472650+0000","last_undegraded":"2023-12-16T21:42:42.472650+0000","last_fullsized":"2023-12-16T21:42:42.472650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472291+0000","last_change":"2023-12-16T21:42:41.384406+0000","last_active":"2023-12-16T21:42:42.472291+0000","last_peered":"2023-12-16T21:42:42.472291+0000","last_clean":"2023-12-16T21:42:42.472291+0000","last_became_active":"2023-12-16T21:42:41.383814+0000","last_became_peered":"2023-12-16T21:42:41.383814+0000","last_unstale":"2023-12-16T21:42:42.472291+0000","last_undegraded":"2023-12-16T21:42:42.472291+0000","last_fullsized":"2023-12-16T21:42:42.472291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475081+0000","last_change":"2023-12-16T21:42:40.437381+0000","last_active":"2023-12-16T21:42:42.475081+0000","last_peered":"2023-12-16T21:42:42.475081+0000","last_clean":"2023-12-16T21:42:42.475081+0000","last_became_active":"2023-12-16T21:42:40.436469+0000","last_became_peered":"2023-12-16T21:42:40.436469+0000","last_unstale":"2023-12-16T21:42:42.475081+0000","last_undegraded":"2023-12-16T21:42:42.475081+0000","last_fullsized":"2023-12-16T21:42:42.475081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471777+0000","last_change":"2023-12-16T21:42:40.432572+0000","last_active":"2023-12-16T21:42:42.471777+0000","last_peered":"2023-12-16T21:42:42.471777+0000","last_clean":"2023-12-16T21:42:42.471777+0000","last_became_active":"2023-12-16T21:42:40.431973+0000","last_became_peered":"2023-12-16T21:42:40.431973+0000","last_unstale":"2023-12-16T21:42:42.471777+0000","last_undegraded":"2023-12-16T21:42:42.471777+0000","last_fullsized":"2023-12-16T21:42:42.471777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472032+0000","last_change":"2023-12-16T21:42:41.372173+0000","last_active":"2023-12-16T21:42:42.472032+0000","last_peered":"2023-12-16T21:42:42.472032+0000","last_clean":"2023-12-16T21:42:42.472032+0000","last_became_active":"2023-12-16T21:42:41.371362+0000","last_became_peered":"2023-12-16T21:42:41.371362+0000","last_unstale":"2023-12-16T21:42:42.472032+0000","last_undegraded":"2023-12-16T21:42:42.472032+0000","last_fullsized":"2023-12-16T21:42:42.472032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474354+0000","last_change":"2023-12-16T21:42:40.458568+0000","last_active":"2023-12-16T21:42:42.474354+0000","last_peered":"2023-12-16T21:42:42.474354+0000","last_clean":"2023-12-16T21:42:42.474354+0000","last_became_active":"2023-12-16T21:42:40.457780+0000","last_became_peered":"2023-12-16T21:42:40.457780+0000","last_unstale":"2023-12-16T21:42:42.474354+0000","last_undegraded":"2023-12-16T21:42:42.474354+0000","last_fullsized":"2023-12-16T21:42:42.474354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329203+0000","last_change":"2023-12-16T21:42:41.367605+0000","last_active":"2023-12-16T21:42:42.329203+0000","last_peered":"2023-12-16T21:42:42.329203+0000","last_clean":"2023-12-16T21:42:42.329203+0000","last_became_active":"2023-12-16T21:42:41.366781+0000","last_became_peered":"2023-12-16T21:42:41.366781+0000","last_unstale":"2023-12-16T21:42:42.329203+0000","last_undegraded":"2023-12-16T21:42:42.329203+0000","last_fullsized":"2023-12-16T21:42:42.329203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475185+0000","last_change":"2023-12-16T21:42:40.462161+0000","last_active":"2023-12-16T21:42:42.475185+0000","last_peered":"2023-12-16T21:42:42.475185+0000","last_clean":"2023-12-16T21:42:42.475185+0000","last_became_active":"2023-12-16T21:42:40.461535+0000","last_became_peered":"2023-12-16T21:42:40.461535+0000","last_unstale":"2023-12-16T21:42:42.475185+0000","last_undegraded":"2023-12-16T21:42:42.475185+0000","last_fullsized":"2023-12-16T21:42:42.475185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476429+0000","last_change":"2023-12-16T21:42:41.366419+0000","last_active":"2023-12-16T21:42:42.476429+0000","last_peered":"2023-12-16T21:42:42.476429+0000","last_clean":"2023-12-16T21:42:42.476429+0000","last_became_active":"2023-12-16T21:42:41.365644+0000","last_became_peered":"2023-12-16T21:42:41.365644+0000","last_unstale":"2023-12-16T21:42:42.476429+0000","last_undegraded":"2023-12-16T21:42:42.476429+0000","last_fullsized":"2023-12-16T21:42:42.476429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476308+0000","last_change":"2023-12-16T21:42:40.461870+0000","last_active":"2023-12-16T21:42:42.476308+0000","last_peered":"2023-12-16T21:42:42.476308+0000","last_clean":"2023-12-16T21:42:42.476308+0000","last_became_active":"2023-12-16T21:42:40.461189+0000","last_became_peered":"2023-12-16T21:42:40.461189+0000","last_unstale":"2023-12-16T21:42:42.476308+0000","last_undegraded":"2023-12-16T21:42:42.476308+0000","last_fullsized":"2023-12-16T21:42:42.476308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471926+0000","last_change":"2023-12-16T21:42:41.385522+0000","last_active":"2023-12-16T21:42:42.471926+0000","last_peered":"2023-12-16T21:42:42.471926+0000","last_clean":"2023-12-16T21:42:42.471926+0000","last_became_active":"2023-12-16T21:42:41.384952+0000","last_became_peered":"2023-12-16T21:42:41.384952+0000","last_unstale":"2023-12-16T21:42:42.471926+0000","last_undegraded":"2023-12-16T21:42:42.471926+0000","last_fullsized":"2023-12-16T21:42:42.471926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330102+0000","last_change":"2023-12-16T21:42:40.453133+0000","last_active":"2023-12-16T21:42:42.330102+0000","last_peered":"2023-12-16T21:42:42.330102+0000","last_clean":"2023-12-16T21:42:42.330102+0000","last_became_active":"2023-12-16T21:42:40.452814+0000","last_became_peered":"2023-12-16T21:42:40.452814+0000","last_unstale":"2023-12-16T21:42:42.330102+0000","last_undegraded":"2023-12-16T21:42:42.330102+0000","last_fullsized":"2023-12-16T21:42:42.330102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335567+0000","last_change":"2023-12-16T21:42:41.359165+0000","last_active":"2023-12-16T21:42:42.335567+0000","last_peered":"2023-12-16T21:42:42.335567+0000","last_clean":"2023-12-16T21:42:42.335567+0000","last_became_active":"2023-12-16T21:42:41.356738+0000","last_became_peered":"2023-12-16T21:42:41.356738+0000","last_unstale":"2023-12-16T21:42:42.335567+0000","last_undegraded":"2023-12-16T21:42:42.335567+0000","last_fullsized":"2023-12-16T21:42:42.335567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475530+0000","last_change":"2023-12-16T21:42:40.428771+0000","last_active":"2023-12-16T21:42:42.475530+0000","last_peered":"2023-12-16T21:42:42.475530+0000","last_clean":"2023-12-16T21:42:42.475530+0000","last_became_active":"2023-12-16T21:42:40.427921+0000","last_became_peered":"2023-12-16T21:42:40.427921+0000","last_unstale":"2023-12-16T21:42:42.475530+0000","last_undegraded":"2023-12-16T21:42:42.475530+0000","last_fullsized":"2023-12-16T21:42:42.475530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334711+0000","last_change":"2023-12-16T21:42:41.373637+0000","last_active":"2023-12-16T21:42:42.334711+0000","last_peered":"2023-12-16T21:42:42.334711+0000","last_clean":"2023-12-16T21:42:42.334711+0000","last_became_active":"2023-12-16T21:42:41.373003+0000","last_became_peered":"2023-12-16T21:42:41.373003+0000","last_unstale":"2023-12-16T21:42:42.334711+0000","last_undegraded":"2023-12-16T21:42:42.334711+0000","last_fullsized":"2023-12-16T21:42:42.334711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:48:55.034570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472336+0000","last_change":"2023-12-16T21:42:40.424774+0000","last_active":"2023-12-16T21:42:42.472336+0000","last_peered":"2023-12-16T21:42:42.472336+0000","last_clean":"2023-12-16T21:42:42.472336+0000","last_became_active":"2023-12-16T21:42:40.424333+0000","last_became_peered":"2023-12-16T21:42:40.424333+0000","last_unstale":"2023-12-16T21:42:42.472336+0000","last_undegraded":"2023-12-16T21:42:42.472336+0000","last_fullsized":"2023-12-16T21:42:42.472336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472450+0000","last_change":"2023-12-16T21:42:41.377224+0000","last_active":"2023-12-16T21:42:42.472450+0000","last_peered":"2023-12-16T21:42:42.472450+0000","last_clean":"2023-12-16T21:42:42.472450+0000","last_became_active":"2023-12-16T21:42:41.376574+0000","last_became_peered":"2023-12-16T21:42:41.376574+0000","last_unstale":"2023-12-16T21:42:42.472450+0000","last_undegraded":"2023-12-16T21:42:42.472450+0000","last_fullsized":"2023-12-16T21:42:42.472450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475529+0000","last_change":"2023-12-16T21:42:40.453599+0000","last_active":"2023-12-16T21:42:42.475529+0000","last_peered":"2023-12-16T21:42:42.475529+0000","last_clean":"2023-12-16T21:42:42.475529+0000","last_became_active":"2023-12-16T21:42:40.452662+0000","last_became_peered":"2023-12-16T21:42:40.452662+0000","last_unstale":"2023-12-16T21:42:42.475529+0000","last_undegraded":"2023-12-16T21:42:42.475529+0000","last_fullsized":"2023-12-16T21:42:42.475529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:02.018716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333919+0000","last_change":"2023-12-16T21:42:41.365750+0000","last_active":"2023-12-16T21:42:42.333919+0000","last_peered":"2023-12-16T21:42:42.333919+0000","last_clean":"2023-12-16T21:42:42.333919+0000","last_became_active":"2023-12-16T21:42:41.365097+0000","last_became_peered":"2023-12-16T21:42:41.365097+0000","last_unstale":"2023-12-16T21:42:42.333919+0000","last_undegraded":"2023-12-16T21:42:42.333919+0000","last_fullsized":"2023-12-16T21:42:42.333919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474655+0000","last_change":"2023-12-16T21:42:40.437256+0000","last_active":"2023-12-16T21:42:42.474655+0000","last_peered":"2023-12-16T21:42:42.474655+0000","last_clean":"2023-12-16T21:42:42.474655+0000","last_became_active":"2023-12-16T21:42:40.436394+0000","last_became_peered":"2023-12-16T21:42:40.436394+0000","last_unstale":"2023-12-16T21:42:42.474655+0000","last_undegraded":"2023-12-16T21:42:42.474655+0000","last_fullsized":"2023-12-16T21:42:42.474655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473943+0000","last_change":"2023-12-16T21:42:41.373882+0000","last_active":"2023-12-16T21:42:42.473943+0000","last_peered":"2023-12-16T21:42:42.473943+0000","last_clean":"2023-12-16T21:42:42.473943+0000","last_became_active":"2023-12-16T21:42:41.373250+0000","last_became_peered":"2023-12-16T21:42:41.373250+0000","last_unstale":"2023-12-16T21:42:42.473943+0000","last_undegraded":"2023-12-16T21:42:42.473943+0000","last_fullsized":"2023-12-16T21:42:42.473943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471379+0000","last_change":"2023-12-16T21:42:40.428292+0000","last_active":"2023-12-16T21:42:42.471379+0000","last_peered":"2023-12-16T21:42:42.471379+0000","last_clean":"2023-12-16T21:42:42.471379+0000","last_became_active":"2023-12-16T21:42:40.427611+0000","last_became_peered":"2023-12-16T21:42:40.427611+0000","last_unstale":"2023-12-16T21:42:42.471379+0000","last_undegraded":"2023-12-16T21:42:42.471379+0000","last_fullsized":"2023-12-16T21:42:42.471379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332822+0000","last_change":"2023-12-16T21:42:41.364395+0000","last_active":"2023-12-16T21:42:42.332822+0000","last_peered":"2023-12-16T21:42:42.332822+0000","last_clean":"2023-12-16T21:42:42.332822+0000","last_became_active":"2023-12-16T21:42:41.363596+0000","last_became_peered":"2023-12-16T21:42:41.363596+0000","last_unstale":"2023-12-16T21:42:42.332822+0000","last_undegraded":"2023-12-16T21:42:42.332822+0000","last_fullsized":"2023-12-16T21:42:42.332822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335857+0000","last_change":"2023-12-16T21:42:40.430896+0000","last_active":"2023-12-16T21:42:42.335857+0000","last_peered":"2023-12-16T21:42:42.335857+0000","last_clean":"2023-12-16T21:42:42.335857+0000","last_became_active":"2023-12-16T21:42:40.425814+0000","last_became_peered":"2023-12-16T21:42:40.425814+0000","last_unstale":"2023-12-16T21:42:42.335857+0000","last_undegraded":"2023-12-16T21:42:42.335857+0000","last_fullsized":"2023-12-16T21:42:42.335857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:39:34.410203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334498+0000","last_change":"2023-12-16T21:42:41.364963+0000","last_active":"2023-12-16T21:42:42.334498+0000","last_peered":"2023-12-16T21:42:42.334498+0000","last_clean":"2023-12-16T21:42:42.334498+0000","last_became_active":"2023-12-16T21:42:41.364207+0000","last_became_peered":"2023-12-16T21:42:41.364207+0000","last_unstale":"2023-12-16T21:42:42.334498+0000","last_undegraded":"2023-12-16T21:42:42.334498+0000","last_fullsized":"2023-12-16T21:42:42.334498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:11:32.433518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471333+0000","last_change":"2023-12-16T21:42:40.424410+0000","last_active":"2023-12-16T21:42:42.471333+0000","last_peered":"2023-12-16T21:42:42.471333+0000","last_clean":"2023-12-16T21:42:42.471333+0000","last_became_active":"2023-12-16T21:42:40.423891+0000","last_became_peered":"2023-12-16T21:42:40.423891+0000","last_unstale":"2023-12-16T21:42:42.471333+0000","last_undegraded":"2023-12-16T21:42:42.471333+0000","last_fullsized":"2023-12-16T21:42:42.471333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334300+0000","last_change":"2023-12-16T21:42:41.366442+0000","last_active":"2023-12-16T21:42:42.334300+0000","last_peered":"2023-12-16T21:42:42.334300+0000","last_clean":"2023-12-16T21:42:42.334300+0000","last_became_active":"2023-12-16T21:42:41.363759+0000","last_became_peered":"2023-12-16T21:42:41.363759+0000","last_unstale":"2023-12-16T21:42:42.334300+0000","last_undegraded":"2023-12-16T21:42:42.334300+0000","last_fullsized":"2023-12-16T21:42:42.334300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:44:20.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330332+0000","last_change":"2023-12-16T21:42:40.429084+0000","last_active":"2023-12-16T21:42:42.330332+0000","last_peered":"2023-12-16T21:42:42.330332+0000","last_clean":"2023-12-16T21:42:42.330332+0000","last_became_active":"2023-12-16T21:42:40.428576+0000","last_became_peered":"2023-12-16T21:42:40.428576+0000","last_unstale":"2023-12-16T21:42:42.330332+0000","last_undegraded":"2023-12-16T21:42:42.330332+0000","last_fullsized":"2023-12-16T21:42:42.330332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475041+0000","last_change":"2023-12-16T21:42:41.385053+0000","last_active":"2023-12-16T21:42:42.475041+0000","last_peered":"2023-12-16T21:42:42.475041+0000","last_clean":"2023-12-16T21:42:42.475041+0000","last_became_active":"2023-12-16T21:42:41.384196+0000","last_became_peered":"2023-12-16T21:42:41.384196+0000","last_unstale":"2023-12-16T21:42:42.475041+0000","last_undegraded":"2023-12-16T21:42:42.475041+0000","last_fullsized":"2023-12-16T21:42:42.475041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471297+0000","last_change":"2023-12-16T21:42:40.461203+0000","last_active":"2023-12-16T21:42:42.471297+0000","last_peered":"2023-12-16T21:42:42.471297+0000","last_clean":"2023-12-16T21:42:42.471297+0000","last_became_active":"2023-12-16T21:42:40.460912+0000","last_became_peered":"2023-12-16T21:42:40.460912+0000","last_unstale":"2023-12-16T21:42:42.471297+0000","last_undegraded":"2023-12-16T21:42:42.471297+0000","last_fullsized":"2023-12-16T21:42:42.471297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:51.118596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335069+0000","last_change":"2023-12-16T21:42:41.369186+0000","last_active":"2023-12-16T21:42:42.335069+0000","last_peered":"2023-12-16T21:42:42.335069+0000","last_clean":"2023-12-16T21:42:42.335069+0000","last_became_active":"2023-12-16T21:42:41.368337+0000","last_became_peered":"2023-12-16T21:42:41.368337+0000","last_unstale":"2023-12-16T21:42:42.335069+0000","last_undegraded":"2023-12-16T21:42:42.335069+0000","last_fullsized":"2023-12-16T21:42:42.335069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:57:56.730962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-16T21:42:41.331102+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:42:41.331102+0000","last_peered":"2023-12-16T21:42:41.331102+0000","last_clean":"2023-12-16T21:42:41.331102+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:42:41.331102+0000","last_undegraded":"2023-12-16T21:42:41.331102+0000","last_fullsized":"2023-12-16T21:42:41.331102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474341+0000","last_change":"2023-12-16T21:42:40.457329+0000","last_active":"2023-12-16T21:42:42.474341+0000","last_peered":"2023-12-16T21:42:42.474341+0000","last_clean":"2023-12-16T21:42:42.474341+0000","last_became_active":"2023-12-16T21:42:40.453943+0000","last_became_peered":"2023-12-16T21:42:40.453943+0000","last_unstale":"2023-12-16T21:42:42.474341+0000","last_undegraded":"2023-12-16T21:42:42.474341+0000","last_fullsized":"2023-12-16T21:42:42.474341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472035+0000","last_change":"2023-12-16T21:42:41.367983+0000","last_active":"2023-12-16T21:42:42.472035+0000","last_peered":"2023-12-16T21:42:42.472035+0000","last_clean":"2023-12-16T21:42:42.472035+0000","last_became_active":"2023-12-16T21:42:41.367201+0000","last_became_peered":"2023-12-16T21:42:41.367201+0000","last_unstale":"2023-12-16T21:42:42.472035+0000","last_undegraded":"2023-12-16T21:42:42.472035+0000","last_fullsized":"2023-12-16T21:42:42.472035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332137+0000","last_change":"2023-12-16T21:42:40.447913+0000","last_active":"2023-12-16T21:42:42.332137+0000","last_peered":"2023-12-16T21:42:42.332137+0000","last_clean":"2023-12-16T21:42:42.332137+0000","last_became_active":"2023-12-16T21:42:40.445110+0000","last_became_peered":"2023-12-16T21:42:40.445110+0000","last_unstale":"2023-12-16T21:42:42.332137+0000","last_undegraded":"2023-12-16T21:42:42.332137+0000","last_fullsized":"2023-12-16T21:42:42.332137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471511+0000","last_change":"2023-12-16T21:42:41.378992+0000","last_active":"2023-12-16T21:42:42.471511+0000","last_peered":"2023-12-16T21:42:42.471511+0000","last_clean":"2023-12-16T21:42:42.471511+0000","last_became_active":"2023-12-16T21:42:41.378192+0000","last_became_peered":"2023-12-16T21:42:41.378192+0000","last_unstale":"2023-12-16T21:42:42.471511+0000","last_undegraded":"2023-12-16T21:42:42.471511+0000","last_fullsized":"2023-12-16T21:42:42.471511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:42.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472434+0000","last_change":"2023-12-16T21:42:40.424908+0000","last_active":"2023-12-16T21:42:42.472434+0000","last_peered":"2023-12-16T21:42:42.472434+0000","last_clean":"2023-12-16T21:42:42.472434+0000","last_became_active":"2023-12-16T21:42:40.424358+0000","last_became_peered":"2023-12-16T21:42:40.424358+0000","last_unstale":"2023-12-16T21:42:42.472434+0000","last_undegraded":"2023-12-16T21:42:42.472434+0000","last_fullsized":"2023-12-16T21:42:42.472434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329469+0000","last_change":"2023-12-16T21:42:41.352792+0000","last_active":"2023-12-16T21:42:42.329469+0000","last_peered":"2023-12-16T21:42:42.329469+0000","last_clean":"2023-12-16T21:42:42.329469+0000","last_became_active":"2023-12-16T21:42:41.352078+0000","last_became_peered":"2023-12-16T21:42:41.352078+0000","last_unstale":"2023-12-16T21:42:42.329469+0000","last_undegraded":"2023-12-16T21:42:42.329469+0000","last_fullsized":"2023-12-16T21:42:42.329469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475741+0000","last_change":"2023-12-16T21:42:40.457237+0000","last_active":"2023-12-16T21:42:42.475741+0000","last_peered":"2023-12-16T21:42:42.475741+0000","last_clean":"2023-12-16T21:42:42.475741+0000","last_became_active":"2023-12-16T21:42:40.452459+0000","last_became_peered":"2023-12-16T21:42:40.452459+0000","last_unstale":"2023-12-16T21:42:42.475741+0000","last_undegraded":"2023-12-16T21:42:42.475741+0000","last_fullsized":"2023-12-16T21:42:42.475741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:57.940854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334555+0000","last_change":"2023-12-16T21:42:41.352263+0000","last_active":"2023-12-16T21:42:42.334555+0000","last_peered":"2023-12-16T21:42:42.334555+0000","last_clean":"2023-12-16T21:42:42.334555+0000","last_became_active":"2023-12-16T21:42:41.347293+0000","last_became_peered":"2023-12-16T21:42:41.347293+0000","last_unstale":"2023-12-16T21:42:42.334555+0000","last_undegraded":"2023-12-16T21:42:42.334555+0000","last_fullsized":"2023-12-16T21:42:42.334555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:52.699335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329677+0000","last_change":"2023-12-16T21:42:40.423235+0000","last_active":"2023-12-16T21:42:42.329677+0000","last_peered":"2023-12-16T21:42:42.329677+0000","last_clean":"2023-12-16T21:42:42.329677+0000","last_became_active":"2023-12-16T21:42:40.422630+0000","last_became_peered":"2023-12-16T21:42:40.422630+0000","last_unstale":"2023-12-16T21:42:42.329677+0000","last_undegraded":"2023-12-16T21:42:42.329677+0000","last_fullsized":"2023-12-16T21:42:42.329677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475779+0000","last_change":"2023-12-16T21:42:41.372994+0000","last_active":"2023-12-16T21:42:42.475779+0000","last_peered":"2023-12-16T21:42:42.475779+0000","last_clean":"2023-12-16T21:42:42.475779+0000","last_became_active":"2023-12-16T21:42:41.372135+0000","last_became_peered":"2023-12-16T21:42:41.372135+0000","last_unstale":"2023-12-16T21:42:42.475779+0000","last_undegraded":"2023-12-16T21:42:42.475779+0000","last_fullsized":"2023-12-16T21:42:42.475779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335404+0000","last_change":"2023-12-16T21:42:40.445536+0000","last_active":"2023-12-16T21:42:42.335404+0000","last_peered":"2023-12-16T21:42:42.335404+0000","last_clean":"2023-12-16T21:42:42.335404+0000","last_became_active":"2023-12-16T21:42:40.444652+0000","last_became_peered":"2023-12-16T21:42:40.444652+0000","last_unstale":"2023-12-16T21:42:42.335404+0000","last_undegraded":"2023-12-16T21:42:42.335404+0000","last_fullsized":"2023-12-16T21:42:42.335404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:38.836527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329548+0000","last_change":"2023-12-16T21:42:41.352754+0000","last_active":"2023-12-16T21:42:42.329548+0000","last_peered":"2023-12-16T21:42:42.329548+0000","last_clean":"2023-12-16T21:42:42.329548+0000","last_became_active":"2023-12-16T21:42:41.351903+0000","last_became_peered":"2023-12-16T21:42:41.351903+0000","last_unstale":"2023-12-16T21:42:42.329548+0000","last_undegraded":"2023-12-16T21:42:42.329548+0000","last_fullsized":"2023-12-16T21:42:42.329548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329921+0000","last_change":"2023-12-16T21:42:40.425139+0000","last_active":"2023-12-16T21:42:42.329921+0000","last_peered":"2023-12-16T21:42:42.329921+0000","last_clean":"2023-12-16T21:42:42.329921+0000","last_became_active":"2023-12-16T21:42:40.423820+0000","last_became_peered":"2023-12-16T21:42:40.423820+0000","last_unstale":"2023-12-16T21:42:42.329921+0000","last_undegraded":"2023-12-16T21:42:42.329921+0000","last_fullsized":"2023-12-16T21:42:42.329921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:37.970083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472944+0000","last_change":"2023-12-16T21:42:41.365965+0000","last_active":"2023-12-16T21:42:42.472944+0000","last_peered":"2023-12-16T21:42:42.472944+0000","last_clean":"2023-12-16T21:42:42.472944+0000","last_became_active":"2023-12-16T21:42:41.365191+0000","last_became_peered":"2023-12-16T21:42:41.365191+0000","last_unstale":"2023-12-16T21:42:42.472944+0000","last_undegraded":"2023-12-16T21:42:42.472944+0000","last_fullsized":"2023-12-16T21:42:42.472944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476214+0000","last_change":"2023-12-16T21:42:40.447186+0000","last_active":"2023-12-16T21:42:42.476214+0000","last_peered":"2023-12-16T21:42:42.476214+0000","last_clean":"2023-12-16T21:42:42.476214+0000","last_became_active":"2023-12-16T21:42:40.446374+0000","last_became_peered":"2023-12-16T21:42:40.446374+0000","last_unstale":"2023-12-16T21:42:42.476214+0000","last_undegraded":"2023-12-16T21:42:42.476214+0000","last_fullsized":"2023-12-16T21:42:42.476214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471782+0000","last_change":"2023-12-16T21:42:41.364750+0000","last_active":"2023-12-16T21:42:42.471782+0000","last_peered":"2023-12-16T21:42:42.471782+0000","last_clean":"2023-12-16T21:42:42.471782+0000","last_became_active":"2023-12-16T21:42:41.363976+0000","last_became_peered":"2023-12-16T21:42:41.363976+0000","last_unstale":"2023-12-16T21:42:42.471782+0000","last_undegraded":"2023-12-16T21:42:42.471782+0000","last_fullsized":"2023-12-16T21:42:42.471782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332140+0000","last_change":"2023-12-16T21:42:40.444481+0000","last_active":"2023-12-16T21:42:42.332140+0000","last_peered":"2023-12-16T21:42:42.332140+0000","last_clean":"2023-12-16T21:42:42.332140+0000","last_became_active":"2023-12-16T21:42:40.443665+0000","last_became_peered":"2023-12-16T21:42:40.443665+0000","last_unstale":"2023-12-16T21:42:42.332140+0000","last_undegraded":"2023-12-16T21:42:42.332140+0000","last_fullsized":"2023-12-16T21:42:42.332140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334181+0000","last_change":"2023-12-16T21:42:41.360039+0000","last_active":"2023-12-16T21:42:42.334181+0000","last_peered":"2023-12-16T21:42:42.334181+0000","last_clean":"2023-12-16T21:42:42.334181+0000","last_became_active":"2023-12-16T21:42:41.359132+0000","last_became_peered":"2023-12-16T21:42:41.359132+0000","last_unstale":"2023-12-16T21:42:42.334181+0000","last_undegraded":"2023-12-16T21:42:42.334181+0000","last_fullsized":"2023-12-16T21:42:42.334181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:52.703815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329301+0000","last_change":"2023-12-16T21:42:40.438910+0000","last_active":"2023-12-16T21:42:42.329301+0000","last_peered":"2023-12-16T21:42:42.329301+0000","last_clean":"2023-12-16T21:42:42.329301+0000","last_became_active":"2023-12-16T21:42:40.438462+0000","last_became_peered":"2023-12-16T21:42:40.438462+0000","last_unstale":"2023-12-16T21:42:42.329301+0000","last_undegraded":"2023-12-16T21:42:42.329301+0000","last_fullsized":"2023-12-16T21:42:42.329301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472184+0000","last_change":"2023-12-16T21:42:41.364881+0000","last_active":"2023-12-16T21:42:42.472184+0000","last_peered":"2023-12-16T21:42:42.472184+0000","last_clean":"2023-12-16T21:42:42.472184+0000","last_became_active":"2023-12-16T21:42:41.363545+0000","last_became_peered":"2023-12-16T21:42:41.363545+0000","last_unstale":"2023-12-16T21:42:42.472184+0000","last_undegraded":"2023-12-16T21:42:42.472184+0000","last_fullsized":"2023-12-16T21:42:42.472184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329671+0000","last_change":"2023-12-16T21:42:40.418011+0000","last_active":"2023-12-16T21:42:42.329671+0000","last_peered":"2023-12-16T21:42:42.329671+0000","last_clean":"2023-12-16T21:42:42.329671+0000","last_became_active":"2023-12-16T21:42:40.417587+0000","last_became_peered":"2023-12-16T21:42:40.417587+0000","last_unstale":"2023-12-16T21:42:42.329671+0000","last_undegraded":"2023-12-16T21:42:42.329671+0000","last_fullsized":"2023-12-16T21:42:42.329671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472662+0000","last_change":"2023-12-16T21:42:41.380236+0000","last_active":"2023-12-16T21:42:42.472662+0000","last_peered":"2023-12-16T21:42:42.472662+0000","last_clean":"2023-12-16T21:42:42.472662+0000","last_became_active":"2023-12-16T21:42:41.379681+0000","last_became_peered":"2023-12-16T21:42:41.379681+0000","last_unstale":"2023-12-16T21:42:42.472662+0000","last_undegraded":"2023-12-16T21:42:42.472662+0000","last_fullsized":"2023-12-16T21:42:42.472662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334158+0000","last_change":"2023-12-16T21:42:40.448985+0000","last_active":"2023-12-16T21:42:42.334158+0000","last_peered":"2023-12-16T21:42:42.334158+0000","last_clean":"2023-12-16T21:42:42.334158+0000","last_became_active":"2023-12-16T21:42:40.443742+0000","last_became_peered":"2023-12-16T21:42:40.443742+0000","last_unstale":"2023-12-16T21:42:42.334158+0000","last_undegraded":"2023-12-16T21:42:42.334158+0000","last_fullsized":"2023-12-16T21:42:42.334158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:53.161999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474831+0000","last_change":"2023-12-16T21:42:41.363951+0000","last_active":"2023-12-16T21:42:42.474831+0000","last_peered":"2023-12-16T21:42:42.474831+0000","last_clean":"2023-12-16T21:42:42.474831+0000","last_became_active":"2023-12-16T21:42:41.363161+0000","last_became_peered":"2023-12-16T21:42:41.363161+0000","last_unstale":"2023-12-16T21:42:42.474831+0000","last_undegraded":"2023-12-16T21:42:42.474831+0000","last_fullsized":"2023-12-16T21:42:42.474831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475088+0000","last_change":"2023-12-16T21:42:41.386385+0000","last_active":"2023-12-16T21:42:42.475088+0000","last_peered":"2023-12-16T21:42:42.475088+0000","last_clean":"2023-12-16T21:42:42.475088+0000","last_became_active":"2023-12-16T21:42:41.385339+0000","last_became_peered":"2023-12-16T21:42:41.385339+0000","last_unstale":"2023-12-16T21:42:42.475088+0000","last_undegraded":"2023-12-16T21:42:42.475088+0000","last_fullsized":"2023-12-16T21:42:42.475088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332799+0000","last_change":"2023-12-16T21:42:40.462751+0000","last_active":"2023-12-16T21:42:42.332799+0000","last_peered":"2023-12-16T21:42:42.332799+0000","last_clean":"2023-12-16T21:42:42.332799+0000","last_became_active":"2023-12-16T21:42:40.461936+0000","last_became_peered":"2023-12-16T21:42:40.461936+0000","last_unstale":"2023-12-16T21:42:42.332799+0000","last_undegraded":"2023-12-16T21:42:42.332799+0000","last_fullsized":"2023-12-16T21:42:42.332799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474683+0000","last_change":"2023-12-16T21:42:41.384715+0000","last_active":"2023-12-16T21:42:42.474683+0000","last_peered":"2023-12-16T21:42:42.474683+0000","last_clean":"2023-12-16T21:42:42.474683+0000","last_became_active":"2023-12-16T21:42:41.383596+0000","last_became_peered":"2023-12-16T21:42:41.383596+0000","last_unstale":"2023-12-16T21:42:42.474683+0000","last_undegraded":"2023-12-16T21:42:42.474683+0000","last_fullsized":"2023-12-16T21:42:42.474683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335044+0000","last_change":"2023-12-16T21:42:40.445218+0000","last_active":"2023-12-16T21:42:42.335044+0000","last_peered":"2023-12-16T21:42:42.335044+0000","last_clean":"2023-12-16T21:42:42.335044+0000","last_became_active":"2023-12-16T21:42:40.443892+0000","last_became_peered":"2023-12-16T21:42:40.443892+0000","last_unstale":"2023-12-16T21:42:42.335044+0000","last_undegraded":"2023-12-16T21:42:42.335044+0000","last_fullsized":"2023-12-16T21:42:42.335044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:20.422855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471754+0000","last_change":"2023-12-16T21:42:41.385409+0000","last_active":"2023-12-16T21:42:42.471754+0000","last_peered":"2023-12-16T21:42:42.471754+0000","last_clean":"2023-12-16T21:42:42.471754+0000","last_became_active":"2023-12-16T21:42:41.384925+0000","last_became_peered":"2023-12-16T21:42:41.384925+0000","last_unstale":"2023-12-16T21:42:42.471754+0000","last_undegraded":"2023-12-16T21:42:42.471754+0000","last_fullsized":"2023-12-16T21:42:42.471754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472993+0000","last_change":"2023-12-16T21:42:40.417361+0000","last_active":"2023-12-16T21:42:42.472993+0000","last_peered":"2023-12-16T21:42:42.472993+0000","last_clean":"2023-12-16T21:42:42.472993+0000","last_became_active":"2023-12-16T21:42:40.416759+0000","last_became_peered":"2023-12-16T21:42:40.416759+0000","last_unstale":"2023-12-16T21:42:42.472993+0000","last_undegraded":"2023-12-16T21:42:42.472993+0000","last_fullsized":"2023-12-16T21:42:42.472993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472885+0000","last_change":"2023-12-16T21:42:41.370637+0000","last_active":"2023-12-16T21:42:42.472885+0000","last_peered":"2023-12-16T21:42:42.472885+0000","last_clean":"2023-12-16T21:42:42.472885+0000","last_became_active":"2023-12-16T21:42:41.369929+0000","last_became_peered":"2023-12-16T21:42:41.369929+0000","last_unstale":"2023-12-16T21:42:42.472885+0000","last_undegraded":"2023-12-16T21:42:42.472885+0000","last_fullsized":"2023-12-16T21:42:42.472885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471919+0000","last_change":"2023-12-16T21:42:40.439054+0000","last_active":"2023-12-16T21:42:42.471919+0000","last_peered":"2023-12-16T21:42:42.471919+0000","last_clean":"2023-12-16T21:42:42.471919+0000","last_became_active":"2023-12-16T21:42:40.438547+0000","last_became_peered":"2023-12-16T21:42:40.438547+0000","last_unstale":"2023-12-16T21:42:42.471919+0000","last_undegraded":"2023-12-16T21:42:42.471919+0000","last_fullsized":"2023-12-16T21:42:42.471919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":69281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.619}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":660287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:42:44.422 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:42:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.571+0000 7f0036fcc640 1 Processor -- start 2023-12-16T21:42:44.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.571+0000 7f0036fcc640 1 -- start start 2023-12-16T21:42:44.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0030071d70 0x7f00300b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00300b3f20 0x7f0030148230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00300b0b20 con 0x7f0030148770 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0030148bc0 con 0x7f00300b3f20 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0030072150 con 0x7f0030071d70 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f00300b3f20 0x7f0030148230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0035569640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f0030148770 0x7f003006a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42998/0 (socket says 172.21.15.46:42998) 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0035569640 1 -- 172.21.15.46:0/2674599209 learned_addr learned my addr 172.21.15.46:0/2674599209 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0034d68640 1 --2- 172.21.15.46:0/2674599209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0030071d70 0x7f00300b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002ffff640 1 -- 172.21.15.46:0/2674599209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0030071d70 msgr2=0x7f00300b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002ffff640 1 --2- 172.21.15.46:0/2674599209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0030071d70 0x7f00300b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002ffff640 1 -- 172.21.15.46:0/2674599209 >> v1:172.21.15.46:6789/0 conn(0x7f0030148770 legacy=0x7f003006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002f7fe640 1 -- 172.21.15.46:0/2674599209 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400796251 0 0) 0x7f00300b0b20 con 0x7f0030148770 2023-12-16T21:42:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002ffff640 1 -- 172.21.15.46:0/2674599209 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0030147c50 con 0x7f00300b3f20 2023-12-16T21:42:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002ffff640 1 --2- 172.21.15.46:0/2674599209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00300b3f20 0x7f0030148230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00180097c0 tx=0x7f001801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f034ef8685ec144 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002f7fe640 1 -- 172.21.15.46:0/2674599209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00180ab600 con 0x7f00300b3f20 2023-12-16T21:42:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002f7fe640 1 -- 172.21.15.46:0/2674599209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00180a7050 con 0x7f00300b3f20 2023-12-16T21:42:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f002f7fe640 1 -- 172.21.15.46:0/2674599209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00180ab980 con 0x7f00300b3f20 2023-12-16T21:42:44.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- 172.21.15.46:0/2674599209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00300b3f20 msgr2=0x7f0030148230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 --2- 172.21.15.46:0/2674599209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00300b3f20 0x7f0030148230 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f00180097c0 tx=0x7f001801c3f0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- 172.21.15.46:0/2674599209 shutdown_connections 2023-12-16T21:42:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 --2- 172.21.15.46:0/2674599209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f00300b3f20 0x7f0030148230 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 --2- 172.21.15.46:0/2674599209 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0030071d70 0x7f00300b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- 172.21.15.46:0/2674599209 >> 172.21.15.46:0/2674599209 conn(0x7f00300b91c0 msgr2=0x7f00300b95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- 172.21.15.46:0/2674599209 shutdown_connections 2023-12-16T21:42:44.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.575+0000 7f0036fcc640 1 -- 172.21.15.46:0/2674599209 wait complete. 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 Processor -- start 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 -- start start 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0030071d70 0x7f003014f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00300b3f20 0x7f003014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0030148770 0x7f003014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003013cdd0 con 0x7f0030071d70 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0030137980 con 0x7f0030148770 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003013ca30 con 0x7f00300b3f20 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0030071d70 0x7f003014f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0030071d70 0x7f003014f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44926/0 (socket says 172.21.15.46:44926) 2023-12-16T21:42:44.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 -- 172.21.15.46:0/3800592419 learned_addr learned my addr 172.21.15.46:0/3800592419 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 -- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00300b3f20 msgr2=0x7f003014d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00300b3f20 0x7f003014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 -- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0030148770 msgr2=0x7f003014da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0030148770 0x7f003014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 -- 172.21.15.46:0/3800592419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0018009420 con 0x7f0030071d70 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0034d68640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0030071d70 0x7f003014f8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f002000d830 tx=0x7f002000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f002d7fa640 1 -- 172.21.15.46:0/3800592419 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000b6c0 con 0x7f0030071d70 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f002d7fa640 1 -- 172.21.15.46:0/3800592419 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002000b820 con 0x7f0030071d70 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003014ee40 con 0x7f0030071d70 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f002d7fa640 1 -- 172.21.15.46:0/3800592419 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002000bb90 con 0x7f0030071d70 2023-12-16T21:42:44.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00301420f0 con 0x7f0030071d70 2023-12-16T21:42:44.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00300688f0 con 0x7f0030071d70 2023-12-16T21:42:44.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.579+0000 7f002d7fa640 1 -- 172.21.15.46:0/3800592419 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f002001e460 con 0x7f0030071d70 2023-12-16T21:42:44.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.583+0000 7f002d7fa640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0010066470 0x7f0010068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.583+0000 7f002d7fa640 1 -- 172.21.15.46:0/3800592419 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00200845e0 con 0x7f0030071d70 2023-12-16T21:42:44.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.583+0000 7f002ffff640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0010066470 0x7f0010068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.583+0000 7f002ffff640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0010066470 0x7f0010068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00180ad000 tx=0x7f0018004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.587+0000 7f002d7fa640 1 -- 172.21.15.46:0/3800592419 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f0020051af0 con 0x7f0030071d70 2023-12-16T21:42:44.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.731+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00300b0d70 con 0x7f0010066470 2023-12-16T21:42:44.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.751+0000 7f002d7fa640 1 -- 172.21.15.46:0/3800592419 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264563 (secure 0 0 0) 0x7f00180a4e90 con 0x7f0010066470 2023-12-16T21:42:44.761 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:44.763 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:42:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0010066470 msgr2=0x7f0010068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0010066470 0x7f0010068930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f00180ad000 tx=0x7f0018004740 comp rx=0 tx=0).stop 2023-12-16T21:42:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0030071d70 msgr2=0x7f003014f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0030071d70 0x7f003014f8a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f002000d830 tx=0x7f002000dcf0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 shutdown_connections 2023-12-16T21:42:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0010066470 0x7f0010068930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0030148770 0x7f003014da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f00300b3f20 0x7f003014d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 --2- 172.21.15.46:0/3800592419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0030071d70 0x7f003014f8a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 >> 172.21.15.46:0/3800592419 conn(0x7f00300b91c0 msgr2=0x7f00300b1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 shutdown_connections 2023-12-16T21:42:44.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.755+0000 7f0036fcc640 1 -- 172.21.15.46:0/3800592419 wait complete. 2023-12-16T21:42:44.777 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-16T21:42:44.351441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":101,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235660,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739060,"statfs":{"total":773094113280,"available":772852797440,"internally_reserved":0,"allocated":2891776,"data_stored":1739804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.963078"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333636+0000","last_change":"2023-12-16T21:42:41.368574+0000","last_active":"2023-12-16T21:42:42.333636+0000","last_peered":"2023-12-16T21:42:42.333636+0000","last_clean":"2023-12-16T21:42:42.333636+0000","last_became_active":"2023-12-16T21:42:41.367555+0000","last_became_peered":"2023-12-16T21:42:41.367555+0000","last_unstale":"2023-12-16T21:42:42.333636+0000","last_undegraded":"2023-12-16T21:42:42.333636+0000","last_fullsized":"2023-12-16T21:42:42.333636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329805+0000","last_change":"2023-12-16T21:42:41.344487+0000","last_active":"2023-12-16T21:42:42.329805+0000","last_peered":"2023-12-16T21:42:42.329805+0000","last_clean":"2023-12-16T21:42:42.329805+0000","last_became_active":"2023-12-16T21:42:41.343798+0000","last_became_peered":"2023-12-16T21:42:41.343798+0000","last_unstale":"2023-12-16T21:42:42.329805+0000","last_undegraded":"2023-12-16T21:42:42.329805+0000","last_fullsized":"2023-12-16T21:42:42.329805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471802+0000","last_change":"2023-12-16T21:42:41.364221+0000","last_active":"2023-12-16T21:42:42.471802+0000","last_peered":"2023-12-16T21:42:42.471802+0000","last_clean":"2023-12-16T21:42:42.471802+0000","last_became_active":"2023-12-16T21:42:41.362506+0000","last_became_peered":"2023-12-16T21:42:41.362506+0000","last_unstale":"2023-12-16T21:42:42.471802+0000","last_undegraded":"2023-12-16T21:42:42.471802+0000","last_fullsized":"2023-12-16T21:42:42.471802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474631+0000","last_change":"2023-12-16T21:42:41.384692+0000","last_active":"2023-12-16T21:42:42.474631+0000","last_peered":"2023-12-16T21:42:42.474631+0000","last_clean":"2023-12-16T21:42:42.474631+0000","last_became_active":"2023-12-16T21:42:41.384156+0000","last_became_peered":"2023-12-16T21:42:41.384156+0000","last_unstale":"2023-12-16T21:42:42.474631+0000","last_undegraded":"2023-12-16T21:42:42.474631+0000","last_fullsized":"2023-12-16T21:42:42.474631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333576+0000","last_change":"2023-12-16T21:42:41.355491+0000","last_active":"2023-12-16T21:42:42.333576+0000","last_peered":"2023-12-16T21:42:42.333576+0000","last_clean":"2023-12-16T21:42:42.333576+0000","last_became_active":"2023-12-16T21:42:41.354519+0000","last_became_peered":"2023-12-16T21:42:41.354519+0000","last_unstale":"2023-12-16T21:42:42.333576+0000","last_undegraded":"2023-12-16T21:42:42.333576+0000","last_fullsized":"2023-12-16T21:42:42.333576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474628+0000","last_change":"2023-12-16T21:42:41.367454+0000","last_active":"2023-12-16T21:42:42.474628+0000","last_peered":"2023-12-16T21:42:42.474628+0000","last_clean":"2023-12-16T21:42:42.474628+0000","last_became_active":"2023-12-16T21:42:41.366663+0000","last_became_peered":"2023-12-16T21:42:41.366663+0000","last_unstale":"2023-12-16T21:42:42.474628+0000","last_undegraded":"2023-12-16T21:42:42.474628+0000","last_fullsized":"2023-12-16T21:42:42.474628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332459+0000","last_change":"2023-12-16T21:42:41.375370+0000","last_active":"2023-12-16T21:42:42.332459+0000","last_peered":"2023-12-16T21:42:42.332459+0000","last_clean":"2023-12-16T21:42:42.332459+0000","last_became_active":"2023-12-16T21:42:41.374733+0000","last_became_peered":"2023-12-16T21:42:41.374733+0000","last_unstale":"2023-12-16T21:42:42.332459+0000","last_undegraded":"2023-12-16T21:42:42.332459+0000","last_fullsized":"2023-12-16T21:42:42.332459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471989+0000","last_change":"2023-12-16T21:42:41.385909+0000","last_active":"2023-12-16T21:42:42.471989+0000","last_peered":"2023-12-16T21:42:42.471989+0000","last_clean":"2023-12-16T21:42:42.471989+0000","last_became_active":"2023-12-16T21:42:41.385386+0000","last_became_peered":"2023-12-16T21:42:41.385386+0000","last_unstale":"2023-12-16T21:42:42.471989+0000","last_undegraded":"2023-12-16T21:42:42.471989+0000","last_fullsized":"2023-12-16T21:42:42.471989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333967+0000","last_change":"2023-12-16T21:42:41.374260+0000","last_active":"2023-12-16T21:42:42.333967+0000","last_peered":"2023-12-16T21:42:42.333967+0000","last_clean":"2023-12-16T21:42:42.333967+0000","last_became_active":"2023-12-16T21:42:41.373101+0000","last_became_peered":"2023-12-16T21:42:41.373101+0000","last_unstale":"2023-12-16T21:42:42.333967+0000","last_undegraded":"2023-12-16T21:42:42.333967+0000","last_fullsized":"2023-12-16T21:42:42.333967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475842+0000","last_change":"2023-12-16T21:42:41.386312+0000","last_active":"2023-12-16T21:42:42.475842+0000","last_peered":"2023-12-16T21:42:42.475842+0000","last_clean":"2023-12-16T21:42:42.475842+0000","last_became_active":"2023-12-16T21:42:41.385651+0000","last_became_peered":"2023-12-16T21:42:41.385651+0000","last_unstale":"2023-12-16T21:42:42.475842+0000","last_undegraded":"2023-12-16T21:42:42.475842+0000","last_fullsized":"2023-12-16T21:42:42.475842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475260+0000","last_change":"2023-12-16T21:42:41.385382+0000","last_active":"2023-12-16T21:42:42.475260+0000","last_peered":"2023-12-16T21:42:42.475260+0000","last_clean":"2023-12-16T21:42:42.475260+0000","last_became_active":"2023-12-16T21:42:41.384671+0000","last_became_peered":"2023-12-16T21:42:41.384671+0000","last_unstale":"2023-12-16T21:42:42.475260+0000","last_undegraded":"2023-12-16T21:42:42.475260+0000","last_fullsized":"2023-12-16T21:42:42.475260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335258+0000","last_change":"2023-12-16T21:42:41.373439+0000","last_active":"2023-12-16T21:42:42.335258+0000","last_peered":"2023-12-16T21:42:42.335258+0000","last_clean":"2023-12-16T21:42:42.335258+0000","last_became_active":"2023-12-16T21:42:41.372462+0000","last_became_peered":"2023-12-16T21:42:41.372462+0000","last_unstale":"2023-12-16T21:42:42.335258+0000","last_undegraded":"2023-12-16T21:42:42.335258+0000","last_fullsized":"2023-12-16T21:42:42.335258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471349+0000","last_change":"2023-12-16T21:42:41.366622+0000","last_active":"2023-12-16T21:42:42.471349+0000","last_peered":"2023-12-16T21:42:42.471349+0000","last_clean":"2023-12-16T21:42:42.471349+0000","last_became_active":"2023-12-16T21:42:41.365732+0000","last_became_peered":"2023-12-16T21:42:41.365732+0000","last_unstale":"2023-12-16T21:42:42.471349+0000","last_undegraded":"2023-12-16T21:42:42.471349+0000","last_fullsized":"2023-12-16T21:42:42.471349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472312+0000","last_change":"2023-12-16T21:42:41.347792+0000","last_active":"2023-12-16T21:42:42.472312+0000","last_peered":"2023-12-16T21:42:42.472312+0000","last_clean":"2023-12-16T21:42:42.472312+0000","last_became_active":"2023-12-16T21:42:41.345588+0000","last_became_peered":"2023-12-16T21:42:41.345588+0000","last_unstale":"2023-12-16T21:42:42.472312+0000","last_undegraded":"2023-12-16T21:42:42.472312+0000","last_fullsized":"2023-12-16T21:42:42.472312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329364+0000","last_change":"2023-12-16T21:42:41.343823+0000","last_active":"2023-12-16T21:42:42.329364+0000","last_peered":"2023-12-16T21:42:42.329364+0000","last_clean":"2023-12-16T21:42:42.329364+0000","last_became_active":"2023-12-16T21:42:41.343022+0000","last_became_peered":"2023-12-16T21:42:41.343022+0000","last_unstale":"2023-12-16T21:42:42.329364+0000","last_undegraded":"2023-12-16T21:42:42.329364+0000","last_fullsized":"2023-12-16T21:42:42.329364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474121+0000","last_change":"2023-12-16T21:42:41.383653+0000","last_active":"2023-12-16T21:42:42.474121+0000","last_peered":"2023-12-16T21:42:42.474121+0000","last_clean":"2023-12-16T21:42:42.474121+0000","last_became_active":"2023-12-16T21:42:41.383223+0000","last_became_peered":"2023-12-16T21:42:41.383223+0000","last_unstale":"2023-12-16T21:42:42.474121+0000","last_undegraded":"2023-12-16T21:42:42.474121+0000","last_fullsized":"2023-12-16T21:42:42.474121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476620+0000","last_change":"2023-12-16T21:42:41.366575+0000","last_active":"2023-12-16T21:42:42.476620+0000","last_peered":"2023-12-16T21:42:42.476620+0000","last_clean":"2023-12-16T21:42:42.476620+0000","last_became_active":"2023-12-16T21:42:41.365758+0000","last_became_peered":"2023-12-16T21:42:41.365758+0000","last_unstale":"2023-12-16T21:42:42.476620+0000","last_undegraded":"2023-12-16T21:42:42.476620+0000","last_fullsized":"2023-12-16T21:42:42.476620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471210+0000","last_change":"2023-12-16T21:42:41.371042+0000","last_active":"2023-12-16T21:42:42.471210+0000","last_peered":"2023-12-16T21:42:42.471210+0000","last_clean":"2023-12-16T21:42:42.471210+0000","last_became_active":"2023-12-16T21:42:41.370204+0000","last_became_peered":"2023-12-16T21:42:41.370204+0000","last_unstale":"2023-12-16T21:42:42.471210+0000","last_undegraded":"2023-12-16T21:42:42.471210+0000","last_fullsized":"2023-12-16T21:42:42.471210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335291+0000","last_change":"2023-12-16T21:42:41.360173+0000","last_active":"2023-12-16T21:42:42.335291+0000","last_peered":"2023-12-16T21:42:42.335291+0000","last_clean":"2023-12-16T21:42:42.335291+0000","last_became_active":"2023-12-16T21:42:41.359373+0000","last_became_peered":"2023-12-16T21:42:41.359373+0000","last_unstale":"2023-12-16T21:42:42.335291+0000","last_undegraded":"2023-12-16T21:42:42.335291+0000","last_fullsized":"2023-12-16T21:42:42.335291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:59:44.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475288+0000","last_change":"2023-12-16T21:42:41.365867+0000","last_active":"2023-12-16T21:42:42.475288+0000","last_peered":"2023-12-16T21:42:42.475288+0000","last_clean":"2023-12-16T21:42:42.475288+0000","last_became_active":"2023-12-16T21:42:41.365202+0000","last_became_peered":"2023-12-16T21:42:41.365202+0000","last_unstale":"2023-12-16T21:42:42.475288+0000","last_undegraded":"2023-12-16T21:42:42.475288+0000","last_fullsized":"2023-12-16T21:42:42.475288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471522+0000","last_change":"2023-12-16T21:42:41.377468+0000","last_active":"2023-12-16T21:42:42.471522+0000","last_peered":"2023-12-16T21:42:42.471522+0000","last_clean":"2023-12-16T21:42:42.471522+0000","last_became_active":"2023-12-16T21:42:41.376856+0000","last_became_peered":"2023-12-16T21:42:41.376856+0000","last_unstale":"2023-12-16T21:42:42.471522+0000","last_undegraded":"2023-12-16T21:42:42.471522+0000","last_fullsized":"2023-12-16T21:42:42.471522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:55.715961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475667+0000","last_change":"2023-12-16T21:42:41.366088+0000","last_active":"2023-12-16T21:42:42.475667+0000","last_peered":"2023-12-16T21:42:42.475667+0000","last_clean":"2023-12-16T21:42:42.475667+0000","last_became_active":"2023-12-16T21:42:41.365380+0000","last_became_peered":"2023-12-16T21:42:41.365380+0000","last_unstale":"2023-12-16T21:42:42.475667+0000","last_undegraded":"2023-12-16T21:42:42.475667+0000","last_fullsized":"2023-12-16T21:42:42.475667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471681+0000","last_change":"2023-12-16T21:42:41.378284+0000","last_active":"2023-12-16T21:42:42.471681+0000","last_peered":"2023-12-16T21:42:42.471681+0000","last_clean":"2023-12-16T21:42:42.471681+0000","last_became_active":"2023-12-16T21:42:41.377070+0000","last_became_peered":"2023-12-16T21:42:41.377070+0000","last_unstale":"2023-12-16T21:42:42.471681+0000","last_undegraded":"2023-12-16T21:42:42.471681+0000","last_fullsized":"2023-12-16T21:42:42.471681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475272+0000","last_change":"2023-12-16T21:42:41.376780+0000","last_active":"2023-12-16T21:42:42.475272+0000","last_peered":"2023-12-16T21:42:42.475272+0000","last_clean":"2023-12-16T21:42:42.475272+0000","last_became_active":"2023-12-16T21:42:41.376224+0000","last_became_peered":"2023-12-16T21:42:41.376224+0000","last_unstale":"2023-12-16T21:42:42.475272+0000","last_undegraded":"2023-12-16T21:42:42.475272+0000","last_fullsized":"2023-12-16T21:42:42.475272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472816+0000","last_change":"2023-12-16T21:42:41.372281+0000","last_active":"2023-12-16T21:42:42.472816+0000","last_peered":"2023-12-16T21:42:42.472816+0000","last_clean":"2023-12-16T21:42:42.472816+0000","last_became_active":"2023-12-16T21:42:41.371551+0000","last_became_peered":"2023-12-16T21:42:41.371551+0000","last_unstale":"2023-12-16T21:42:42.472816+0000","last_undegraded":"2023-12-16T21:42:42.472816+0000","last_fullsized":"2023-12-16T21:42:42.472816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475958+0000","last_change":"2023-12-16T21:42:41.375672+0000","last_active":"2023-12-16T21:42:42.475958+0000","last_peered":"2023-12-16T21:42:42.475958+0000","last_clean":"2023-12-16T21:42:42.475958+0000","last_became_active":"2023-12-16T21:42:41.370826+0000","last_became_peered":"2023-12-16T21:42:41.370826+0000","last_unstale":"2023-12-16T21:42:42.475958+0000","last_undegraded":"2023-12-16T21:42:42.475958+0000","last_fullsized":"2023-12-16T21:42:42.475958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329872+0000","last_change":"2023-12-16T21:42:41.368001+0000","last_active":"2023-12-16T21:42:42.329872+0000","last_peered":"2023-12-16T21:42:42.329872+0000","last_clean":"2023-12-16T21:42:42.329872+0000","last_became_active":"2023-12-16T21:42:41.367168+0000","last_became_peered":"2023-12-16T21:42:41.367168+0000","last_unstale":"2023-12-16T21:42:42.329872+0000","last_undegraded":"2023-12-16T21:42:42.329872+0000","last_fullsized":"2023-12-16T21:42:42.329872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472910+0000","last_change":"2023-12-16T21:42:41.377845+0000","last_active":"2023-12-16T21:42:42.472910+0000","last_peered":"2023-12-16T21:42:42.472910+0000","last_clean":"2023-12-16T21:42:42.472910+0000","last_became_active":"2023-12-16T21:42:41.377023+0000","last_became_peered":"2023-12-16T21:42:41.377023+0000","last_unstale":"2023-12-16T21:42:42.472910+0000","last_undegraded":"2023-12-16T21:42:42.472910+0000","last_fullsized":"2023-12-16T21:42:42.472910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472206+0000","last_change":"2023-12-16T21:42:41.371637+0000","last_active":"2023-12-16T21:42:42.472206+0000","last_peered":"2023-12-16T21:42:42.472206+0000","last_clean":"2023-12-16T21:42:42.472206+0000","last_became_active":"2023-12-16T21:42:41.370853+0000","last_became_peered":"2023-12-16T21:42:41.370853+0000","last_unstale":"2023-12-16T21:42:42.472206+0000","last_undegraded":"2023-12-16T21:42:42.472206+0000","last_fullsized":"2023-12-16T21:42:42.472206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475286+0000","last_change":"2023-12-16T21:42:41.375235+0000","last_active":"2023-12-16T21:42:42.475286+0000","last_peered":"2023-12-16T21:42:42.475286+0000","last_clean":"2023-12-16T21:42:42.475286+0000","last_became_active":"2023-12-16T21:42:41.374570+0000","last_became_peered":"2023-12-16T21:42:41.374570+0000","last_unstale":"2023-12-16T21:42:42.475286+0000","last_undegraded":"2023-12-16T21:42:42.475286+0000","last_fullsized":"2023-12-16T21:42:42.475286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472367+0000","last_change":"2023-12-16T21:42:41.383749+0000","last_active":"2023-12-16T21:42:42.472367+0000","last_peered":"2023-12-16T21:42:42.472367+0000","last_clean":"2023-12-16T21:42:42.472367+0000","last_became_active":"2023-12-16T21:42:41.383009+0000","last_became_peered":"2023-12-16T21:42:41.383009+0000","last_unstale":"2023-12-16T21:42:42.472367+0000","last_undegraded":"2023-12-16T21:42:42.472367+0000","last_fullsized":"2023-12-16T21:42:42.472367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473979+0000","last_change":"2023-12-16T21:42:41.379801+0000","last_active":"2023-12-16T21:42:42.473979+0000","last_peered":"2023-12-16T21:42:42.473979+0000","last_clean":"2023-12-16T21:42:42.473979+0000","last_became_active":"2023-12-16T21:42:41.379000+0000","last_became_peered":"2023-12-16T21:42:41.379000+0000","last_unstale":"2023-12-16T21:42:42.473979+0000","last_undegraded":"2023-12-16T21:42:42.473979+0000","last_fullsized":"2023-12-16T21:42:42.473979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333784+0000","last_change":"2023-12-16T21:42:41.351207+0000","last_active":"2023-12-16T21:42:42.333784+0000","last_peered":"2023-12-16T21:42:42.333784+0000","last_clean":"2023-12-16T21:42:42.333784+0000","last_became_active":"2023-12-16T21:42:41.349156+0000","last_became_peered":"2023-12-16T21:42:41.349156+0000","last_unstale":"2023-12-16T21:42:42.333784+0000","last_undegraded":"2023-12-16T21:42:42.333784+0000","last_fullsized":"2023-12-16T21:42:42.333784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:52.130538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472650+0000","last_change":"2023-12-16T21:42:40.442950+0000","last_active":"2023-12-16T21:42:42.472650+0000","last_peered":"2023-12-16T21:42:42.472650+0000","last_clean":"2023-12-16T21:42:42.472650+0000","last_became_active":"2023-12-16T21:42:40.442532+0000","last_became_peered":"2023-12-16T21:42:40.442532+0000","last_unstale":"2023-12-16T21:42:42.472650+0000","last_undegraded":"2023-12-16T21:42:42.472650+0000","last_fullsized":"2023-12-16T21:42:42.472650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472291+0000","last_change":"2023-12-16T21:42:41.384406+0000","last_active":"2023-12-16T21:42:42.472291+0000","last_peered":"2023-12-16T21:42:42.472291+0000","last_clean":"2023-12-16T21:42:42.472291+0000","last_became_active":"2023-12-16T21:42:41.383814+0000","last_became_peered":"2023-12-16T21:42:41.383814+0000","last_unstale":"2023-12-16T21:42:42.472291+0000","last_undegraded":"2023-12-16T21:42:42.472291+0000","last_fullsized":"2023-12-16T21:42:42.472291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475081+0000","last_change":"2023-12-16T21:42:40.437381+0000","last_active":"2023-12-16T21:42:42.475081+0000","last_peered":"2023-12-16T21:42:42.475081+0000","last_clean":"2023-12-16T21:42:42.475081+0000","last_became_active":"2023-12-16T21:42:40.436469+0000","last_became_peered":"2023-12-16T21:42:40.436469+0000","last_unstale":"2023-12-16T21:42:42.475081+0000","last_undegraded":"2023-12-16T21:42:42.475081+0000","last_fullsized":"2023-12-16T21:42:42.475081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471777+0000","last_change":"2023-12-16T21:42:40.432572+0000","last_active":"2023-12-16T21:42:42.471777+0000","last_peered":"2023-12-16T21:42:42.471777+0000","last_clean":"2023-12-16T21:42:42.471777+0000","last_became_active":"2023-12-16T21:42:40.431973+0000","last_became_peered":"2023-12-16T21:42:40.431973+0000","last_unstale":"2023-12-16T21:42:42.471777+0000","last_undegraded":"2023-12-16T21:42:42.471777+0000","last_fullsized":"2023-12-16T21:42:42.471777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472032+0000","last_change":"2023-12-16T21:42:41.372173+0000","last_active":"2023-12-16T21:42:42.472032+0000","last_peered":"2023-12-16T21:42:42.472032+0000","last_clean":"2023-12-16T21:42:42.472032+0000","last_became_active":"2023-12-16T21:42:41.371362+0000","last_became_peered":"2023-12-16T21:42:41.371362+0000","last_unstale":"2023-12-16T21:42:42.472032+0000","last_undegraded":"2023-12-16T21:42:42.472032+0000","last_fullsized":"2023-12-16T21:42:42.472032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474354+0000","last_change":"2023-12-16T21:42:40.458568+0000","last_active":"2023-12-16T21:42:42.474354+0000","last_peered":"2023-12-16T21:42:42.474354+0000","last_clean":"2023-12-16T21:42:42.474354+0000","last_became_active":"2023-12-16T21:42:40.457780+0000","last_became_peered":"2023-12-16T21:42:40.457780+0000","last_unstale":"2023-12-16T21:42:42.474354+0000","last_undegraded":"2023-12-16T21:42:42.474354+0000","last_fullsized":"2023-12-16T21:42:42.474354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329203+0000","last_change":"2023-12-16T21:42:41.367605+0000","last_active":"2023-12-16T21:42:42.329203+0000","last_peered":"2023-12-16T21:42:42.329203+0000","last_clean":"2023-12-16T21:42:42.329203+0000","last_became_active":"2023-12-16T21:42:41.366781+0000","last_became_peered":"2023-12-16T21:42:41.366781+0000","last_unstale":"2023-12-16T21:42:42.329203+0000","last_undegraded":"2023-12-16T21:42:42.329203+0000","last_fullsized":"2023-12-16T21:42:42.329203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475185+0000","last_change":"2023-12-16T21:42:40.462161+0000","last_active":"2023-12-16T21:42:42.475185+0000","last_peered":"2023-12-16T21:42:42.475185+0000","last_clean":"2023-12-16T21:42:42.475185+0000","last_became_active":"2023-12-16T21:42:40.461535+0000","last_became_peered":"2023-12-16T21:42:40.461535+0000","last_unstale":"2023-12-16T21:42:42.475185+0000","last_undegraded":"2023-12-16T21:42:42.475185+0000","last_fullsized":"2023-12-16T21:42:42.475185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476429+0000","last_change":"2023-12-16T21:42:41.366419+0000","last_active":"2023-12-16T21:42:42.476429+0000","last_peered":"2023-12-16T21:42:42.476429+0000","last_clean":"2023-12-16T21:42:42.476429+0000","last_became_active":"2023-12-16T21:42:41.365644+0000","last_became_peered":"2023-12-16T21:42:41.365644+0000","last_unstale":"2023-12-16T21:42:42.476429+0000","last_undegraded":"2023-12-16T21:42:42.476429+0000","last_fullsized":"2023-12-16T21:42:42.476429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476308+0000","last_change":"2023-12-16T21:42:40.461870+0000","last_active":"2023-12-16T21:42:42.476308+0000","last_peered":"2023-12-16T21:42:42.476308+0000","last_clean":"2023-12-16T21:42:42.476308+0000","last_became_active":"2023-12-16T21:42:40.461189+0000","last_became_peered":"2023-12-16T21:42:40.461189+0000","last_unstale":"2023-12-16T21:42:42.476308+0000","last_undegraded":"2023-12-16T21:42:42.476308+0000","last_fullsized":"2023-12-16T21:42:42.476308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471926+0000","last_change":"2023-12-16T21:42:41.385522+0000","last_active":"2023-12-16T21:42:42.471926+0000","last_peered":"2023-12-16T21:42:42.471926+0000","last_clean":"2023-12-16T21:42:42.471926+0000","last_became_active":"2023-12-16T21:42:41.384952+0000","last_became_peered":"2023-12-16T21:42:41.384952+0000","last_unstale":"2023-12-16T21:42:42.471926+0000","last_undegraded":"2023-12-16T21:42:42.471926+0000","last_fullsized":"2023-12-16T21:42:42.471926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330102+0000","last_change":"2023-12-16T21:42:40.453133+0000","last_active":"2023-12-16T21:42:42.330102+0000","last_peered":"2023-12-16T21:42:42.330102+0000","last_clean":"2023-12-16T21:42:42.330102+0000","last_became_active":"2023-12-16T21:42:40.452814+0000","last_became_peered":"2023-12-16T21:42:40.452814+0000","last_unstale":"2023-12-16T21:42:42.330102+0000","last_undegraded":"2023-12-16T21:42:42.330102+0000","last_fullsized":"2023-12-16T21:42:42.330102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335567+0000","last_change":"2023-12-16T21:42:41.359165+0000","last_active":"2023-12-16T21:42:42.335567+0000","last_peered":"2023-12-16T21:42:42.335567+0000","last_clean":"2023-12-16T21:42:42.335567+0000","last_became_active":"2023-12-16T21:42:41.356738+0000","last_became_peered":"2023-12-16T21:42:41.356738+0000","last_unstale":"2023-12-16T21:42:42.335567+0000","last_undegraded":"2023-12-16T21:42:42.335567+0000","last_fullsized":"2023-12-16T21:42:42.335567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475530+0000","last_change":"2023-12-16T21:42:40.428771+0000","last_active":"2023-12-16T21:42:42.475530+0000","last_peered":"2023-12-16T21:42:42.475530+0000","last_clean":"2023-12-16T21:42:42.475530+0000","last_became_active":"2023-12-16T21:42:40.427921+0000","last_became_peered":"2023-12-16T21:42:40.427921+0000","last_unstale":"2023-12-16T21:42:42.475530+0000","last_undegraded":"2023-12-16T21:42:42.475530+0000","last_fullsized":"2023-12-16T21:42:42.475530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334711+0000","last_change":"2023-12-16T21:42:41.373637+0000","last_active":"2023-12-16T21:42:42.334711+0000","last_peered":"2023-12-16T21:42:42.334711+0000","last_clean":"2023-12-16T21:42:42.334711+0000","last_became_active":"2023-12-16T21:42:41.373003+0000","last_became_peered":"2023-12-16T21:42:41.373003+0000","last_unstale":"2023-12-16T21:42:42.334711+0000","last_undegraded":"2023-12-16T21:42:42.334711+0000","last_fullsized":"2023-12-16T21:42:42.334711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:48:55.034570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472336+0000","last_change":"2023-12-16T21:42:40.424774+0000","last_active":"2023-12-16T21:42:42.472336+0000","last_peered":"2023-12-16T21:42:42.472336+0000","last_clean":"2023-12-16T21:42:42.472336+0000","last_became_active":"2023-12-16T21:42:40.424333+0000","last_became_peered":"2023-12-16T21:42:40.424333+0000","last_unstale":"2023-12-16T21:42:42.472336+0000","last_undegraded":"2023-12-16T21:42:42.472336+0000","last_fullsized":"2023-12-16T21:42:42.472336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472450+0000","last_change":"2023-12-16T21:42:41.377224+0000","last_active":"2023-12-16T21:42:42.472450+0000","last_peered":"2023-12-16T21:42:42.472450+0000","last_clean":"2023-12-16T21:42:42.472450+0000","last_became_active":"2023-12-16T21:42:41.376574+0000","last_became_peered":"2023-12-16T21:42:41.376574+0000","last_unstale":"2023-12-16T21:42:42.472450+0000","last_undegraded":"2023-12-16T21:42:42.472450+0000","last_fullsized":"2023-12-16T21:42:42.472450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475529+0000","last_change":"2023-12-16T21:42:40.453599+0000","last_active":"2023-12-16T21:42:42.475529+0000","last_peered":"2023-12-16T21:42:42.475529+0000","last_clean":"2023-12-16T21:42:42.475529+0000","last_became_active":"2023-12-16T21:42:40.452662+0000","last_became_peered":"2023-12-16T21:42:40.452662+0000","last_unstale":"2023-12-16T21:42:42.475529+0000","last_undegraded":"2023-12-16T21:42:42.475529+0000","last_fullsized":"2023-12-16T21:42:42.475529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:02.018716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333919+0000","last_change":"2023-12-16T21:42:41.365750+0000","last_active":"2023-12-16T21:42:42.333919+0000","last_peered":"2023-12-16T21:42:42.333919+0000","last_clean":"2023-12-16T21:42:42.333919+0000","last_became_active":"2023-12-16T21:42:41.365097+0000","last_became_peered":"2023-12-16T21:42:41.365097+0000","last_unstale":"2023-12-16T21:42:42.333919+0000","last_undegraded":"2023-12-16T21:42:42.333919+0000","last_fullsized":"2023-12-16T21:42:42.333919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474655+0000","last_change":"2023-12-16T21:42:40.437256+0000","last_active":"2023-12-16T21:42:42.474655+0000","last_peered":"2023-12-16T21:42:42.474655+0000","last_clean":"2023-12-16T21:42:42.474655+0000","last_became_active":"2023-12-16T21:42:40.436394+0000","last_became_peered":"2023-12-16T21:42:40.436394+0000","last_unstale":"2023-12-16T21:42:42.474655+0000","last_undegraded":"2023-12-16T21:42:42.474655+0000","last_fullsized":"2023-12-16T21:42:42.474655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473943+0000","last_change":"2023-12-16T21:42:41.373882+0000","last_active":"2023-12-16T21:42:42.473943+0000","last_peered":"2023-12-16T21:42:42.473943+0000","last_clean":"2023-12-16T21:42:42.473943+0000","last_became_active":"2023-12-16T21:42:41.373250+0000","last_became_peered":"2023-12-16T21:42:41.373250+0000","last_unstale":"2023-12-16T21:42:42.473943+0000","last_undegraded":"2023-12-16T21:42:42.473943+0000","last_fullsized":"2023-12-16T21:42:42.473943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471379+0000","last_change":"2023-12-16T21:42:40.428292+0000","last_active":"2023-12-16T21:42:42.471379+0000","last_peered":"2023-12-16T21:42:42.471379+0000","last_clean":"2023-12-16T21:42:42.471379+0000","last_became_active":"2023-12-16T21:42:40.427611+0000","last_became_peered":"2023-12-16T21:42:40.427611+0000","last_unstale":"2023-12-16T21:42:42.471379+0000","last_undegraded":"2023-12-16T21:42:42.471379+0000","last_fullsized":"2023-12-16T21:42:42.471379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332822+0000","last_change":"2023-12-16T21:42:41.364395+0000","last_active":"2023-12-16T21:42:42.332822+0000","last_peered":"2023-12-16T21:42:42.332822+0000","last_clean":"2023-12-16T21:42:42.332822+0000","last_became_active":"2023-12-16T21:42:41.363596+0000","last_became_peered":"2023-12-16T21:42:41.363596+0000","last_unstale":"2023-12-16T21:42:42.332822+0000","last_undegraded":"2023-12-16T21:42:42.332822+0000","last_fullsized":"2023-12-16T21:42:42.332822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335857+0000","last_change":"2023-12-16T21:42:40.430896+0000","last_active":"2023-12-16T21:42:42.335857+0000","last_peered":"2023-12-16T21:42:42.335857+0000","last_clean":"2023-12-16T21:42:42.335857+0000","last_became_active":"2023-12-16T21:42:40.425814+0000","last_became_peered":"2023-12-16T21:42:40.425814+0000","last_unstale":"2023-12-16T21:42:42.335857+0000","last_undegraded":"2023-12-16T21:42:42.335857+0000","last_fullsized":"2023-12-16T21:42:42.335857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:39:34.410203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334498+0000","last_change":"2023-12-16T21:42:41.364963+0000","last_active":"2023-12-16T21:42:42.334498+0000","last_peered":"2023-12-16T21:42:42.334498+0000","last_clean":"2023-12-16T21:42:42.334498+0000","last_became_active":"2023-12-16T21:42:41.364207+0000","last_became_peered":"2023-12-16T21:42:41.364207+0000","last_unstale":"2023-12-16T21:42:42.334498+0000","last_undegraded":"2023-12-16T21:42:42.334498+0000","last_fullsized":"2023-12-16T21:42:42.334498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:11:32.433518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471333+0000","last_change":"2023-12-16T21:42:40.424410+0000","last_active":"2023-12-16T21:42:42.471333+0000","last_peered":"2023-12-16T21:42:42.471333+0000","last_clean":"2023-12-16T21:42:42.471333+0000","last_became_active":"2023-12-16T21:42:40.423891+0000","last_became_peered":"2023-12-16T21:42:40.423891+0000","last_unstale":"2023-12-16T21:42:42.471333+0000","last_undegraded":"2023-12-16T21:42:42.471333+0000","last_fullsized":"2023-12-16T21:42:42.471333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334300+0000","last_change":"2023-12-16T21:42:41.366442+0000","last_active":"2023-12-16T21:42:42.334300+0000","last_peered":"2023-12-16T21:42:42.334300+0000","last_clean":"2023-12-16T21:42:42.334300+0000","last_became_active":"2023-12-16T21:42:41.363759+0000","last_became_peered":"2023-12-16T21:42:41.363759+0000","last_unstale":"2023-12-16T21:42:42.334300+0000","last_undegraded":"2023-12-16T21:42:42.334300+0000","last_fullsized":"2023-12-16T21:42:42.334300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:44:20.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330332+0000","last_change":"2023-12-16T21:42:40.429084+0000","last_active":"2023-12-16T21:42:42.330332+0000","last_peered":"2023-12-16T21:42:42.330332+0000","last_clean":"2023-12-16T21:42:42.330332+0000","last_became_active":"2023-12-16T21:42:40.428576+0000","last_became_peered":"2023-12-16T21:42:40.428576+0000","last_unstale":"2023-12-16T21:42:42.330332+0000","last_undegraded":"2023-12-16T21:42:42.330332+0000","last_fullsized":"2023-12-16T21:42:42.330332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475041+0000","last_change":"2023-12-16T21:42:41.385053+0000","last_active":"2023-12-16T21:42:42.475041+0000","last_peered":"2023-12-16T21:42:42.475041+0000","last_clean":"2023-12-16T21:42:42.475041+0000","last_became_active":"2023-12-16T21:42:41.384196+0000","last_became_peered":"2023-12-16T21:42:41.384196+0000","last_unstale":"2023-12-16T21:42:42.475041+0000","last_undegraded":"2023-12-16T21:42:42.475041+0000","last_fullsized":"2023-12-16T21:42:42.475041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471297+0000","last_change":"2023-12-16T21:42:40.461203+0000","last_active":"2023-12-16T21:42:42.471297+0000","last_peered":"2023-12-16T21:42:42.471297+0000","last_clean":"2023-12-16T21:42:42.471297+0000","last_became_active":"2023-12-16T21:42:40.460912+0000","last_became_peered":"2023-12-16T21:42:40.460912+0000","last_unstale":"2023-12-16T21:42:42.471297+0000","last_undegraded":"2023-12-16T21:42:42.471297+0000","last_fullsized":"2023-12-16T21:42:42.471297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:51.118596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335069+0000","last_change":"2023-12-16T21:42:41.369186+0000","last_active":"2023-12-16T21:42:42.335069+0000","last_peered":"2023-12-16T21:42:42.335069+0000","last_clean":"2023-12-16T21:42:42.335069+0000","last_became_active":"2023-12-16T21:42:41.368337+0000","last_became_peered":"2023-12-16T21:42:41.368337+0000","last_unstale":"2023-12-16T21:42:42.335069+0000","last_undegraded":"2023-12-16T21:42:42.335069+0000","last_fullsized":"2023-12-16T21:42:42.335069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:57:56.730962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-16T21:42:41.331102+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:42:41.331102+0000","last_peered":"2023-12-16T21:42:41.331102+0000","last_clean":"2023-12-16T21:42:41.331102+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:42:41.331102+0000","last_undegraded":"2023-12-16T21:42:41.331102+0000","last_fullsized":"2023-12-16T21:42:41.331102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474341+0000","last_change":"2023-12-16T21:42:40.457329+0000","last_active":"2023-12-16T21:42:42.474341+0000","last_peered":"2023-12-16T21:42:42.474341+0000","last_clean":"2023-12-16T21:42:42.474341+0000","last_became_active":"2023-12-16T21:42:40.453943+0000","last_became_peered":"2023-12-16T21:42:40.453943+0000","last_unstale":"2023-12-16T21:42:42.474341+0000","last_undegraded":"2023-12-16T21:42:42.474341+0000","last_fullsized":"2023-12-16T21:42:42.474341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472035+0000","last_change":"2023-12-16T21:42:41.367983+0000","last_active":"2023-12-16T21:42:42.472035+0000","last_peered":"2023-12-16T21:42:42.472035+0000","last_clean":"2023-12-16T21:42:42.472035+0000","last_became_active":"2023-12-16T21:42:41.367201+0000","last_became_peered":"2023-12-16T21:42:41.367201+0000","last_unstale":"2023-12-16T21:42:42.472035+0000","last_undegraded":"2023-12-16T21:42:42.472035+0000","last_fullsized":"2023-12-16T21:42:42.472035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332137+0000","last_change":"2023-12-16T21:42:40.447913+0000","last_active":"2023-12-16T21:42:42.332137+0000","last_peered":"2023-12-16T21:42:42.332137+0000","last_clean":"2023-12-16T21:42:42.332137+0000","last_became_active":"2023-12-16T21:42:40.445110+0000","last_became_peered":"2023-12-16T21:42:40.445110+0000","last_unstale":"2023-12-16T21:42:42.332137+0000","last_undegraded":"2023-12-16T21:42:42.332137+0000","last_fullsized":"2023-12-16T21:42:42.332137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471511+0000","last_change":"2023-12-16T21:42:41.378992+0000","last_active":"2023-12-16T21:42:42.471511+0000","last_peered":"2023-12-16T21:42:42.471511+0000","last_clean":"2023-12-16T21:42:42.471511+0000","last_became_active":"2023-12-16T21:42:41.378192+0000","last_became_peered":"2023-12-16T21:42:41.378192+0000","last_unstale":"2023-12-16T21:42:42.471511+0000","last_undegraded":"2023-12-16T21:42:42.471511+0000","last_fullsized":"2023-12-16T21:42:42.471511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:42.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472434+0000","last_change":"2023-12-16T21:42:40.424908+0000","last_active":"2023-12-16T21:42:42.472434+0000","last_peered":"2023-12-16T21:42:42.472434+0000","last_clean":"2023-12-16T21:42:42.472434+0000","last_became_active":"2023-12-16T21:42:40.424358+0000","last_became_peered":"2023-12-16T21:42:40.424358+0000","last_unstale":"2023-12-16T21:42:42.472434+0000","last_undegraded":"2023-12-16T21:42:42.472434+0000","last_fullsized":"2023-12-16T21:42:42.472434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329469+0000","last_change":"2023-12-16T21:42:41.352792+0000","last_active":"2023-12-16T21:42:42.329469+0000","last_peered":"2023-12-16T21:42:42.329469+0000","last_clean":"2023-12-16T21:42:42.329469+0000","last_became_active":"2023-12-16T21:42:41.352078+0000","last_became_peered":"2023-12-16T21:42:41.352078+0000","last_unstale":"2023-12-16T21:42:42.329469+0000","last_undegraded":"2023-12-16T21:42:42.329469+0000","last_fullsized":"2023-12-16T21:42:42.329469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475741+0000","last_change":"2023-12-16T21:42:40.457237+0000","last_active":"2023-12-16T21:42:42.475741+0000","last_peered":"2023-12-16T21:42:42.475741+0000","last_clean":"2023-12-16T21:42:42.475741+0000","last_became_active":"2023-12-16T21:42:40.452459+0000","last_became_peered":"2023-12-16T21:42:40.452459+0000","last_unstale":"2023-12-16T21:42:42.475741+0000","last_undegraded":"2023-12-16T21:42:42.475741+0000","last_fullsized":"2023-12-16T21:42:42.475741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:57.940854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334555+0000","last_change":"2023-12-16T21:42:41.352263+0000","last_active":"2023-12-16T21:42:42.334555+0000","last_peered":"2023-12-16T21:42:42.334555+0000","last_clean":"2023-12-16T21:42:42.334555+0000","last_became_active":"2023-12-16T21:42:41.347293+0000","last_became_peered":"2023-12-16T21:42:41.347293+0000","last_unstale":"2023-12-16T21:42:42.334555+0000","last_undegraded":"2023-12-16T21:42:42.334555+0000","last_fullsized":"2023-12-16T21:42:42.334555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:52.699335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329677+0000","last_change":"2023-12-16T21:42:40.423235+0000","last_active":"2023-12-16T21:42:42.329677+0000","last_peered":"2023-12-16T21:42:42.329677+0000","last_clean":"2023-12-16T21:42:42.329677+0000","last_became_active":"2023-12-16T21:42:40.422630+0000","last_became_peered":"2023-12-16T21:42:40.422630+0000","last_unstale":"2023-12-16T21:42:42.329677+0000","last_undegraded":"2023-12-16T21:42:42.329677+0000","last_fullsized":"2023-12-16T21:42:42.329677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475779+0000","last_change":"2023-12-16T21:42:41.372994+0000","last_active":"2023-12-16T21:42:42.475779+0000","last_peered":"2023-12-16T21:42:42.475779+0000","last_clean":"2023-12-16T21:42:42.475779+0000","last_became_active":"2023-12-16T21:42:41.372135+0000","last_became_peered":"2023-12-16T21:42:41.372135+0000","last_unstale":"2023-12-16T21:42:42.475779+0000","last_undegraded":"2023-12-16T21:42:42.475779+0000","last_fullsized":"2023-12-16T21:42:42.475779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335404+0000","last_change":"2023-12-16T21:42:40.445536+0000","last_active":"2023-12-16T21:42:42.335404+0000","last_peered":"2023-12-16T21:42:42.335404+0000","last_clean":"2023-12-16T21:42:42.335404+0000","last_became_active":"2023-12-16T21:42:40.444652+0000","last_became_peered":"2023-12-16T21:42:40.444652+0000","last_unstale":"2023-12-16T21:42:42.335404+0000","last_undegraded":"2023-12-16T21:42:42.335404+0000","last_fullsized":"2023-12-16T21:42:42.335404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:38.836527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329548+0000","last_change":"2023-12-16T21:42:41.352754+0000","last_active":"2023-12-16T21:42:42.329548+0000","last_peered":"2023-12-16T21:42:42.329548+0000","last_clean":"2023-12-16T21:42:42.329548+0000","last_became_active":"2023-12-16T21:42:41.351903+0000","last_became_peered":"2023-12-16T21:42:41.351903+0000","last_unstale":"2023-12-16T21:42:42.329548+0000","last_undegraded":"2023-12-16T21:42:42.329548+0000","last_fullsized":"2023-12-16T21:42:42.329548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329921+0000","last_change":"2023-12-16T21:42:40.425139+0000","last_active":"2023-12-16T21:42:42.329921+0000","last_peered":"2023-12-16T21:42:42.329921+0000","last_clean":"2023-12-16T21:42:42.329921+0000","last_became_active":"2023-12-16T21:42:40.423820+0000","last_became_peered":"2023-12-16T21:42:40.423820+0000","last_unstale":"2023-12-16T21:42:42.329921+0000","last_undegraded":"2023-12-16T21:42:42.329921+0000","last_fullsized":"2023-12-16T21:42:42.329921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:37.970083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472944+0000","last_change":"2023-12-16T21:42:41.365965+0000","last_active":"2023-12-16T21:42:42.472944+0000","last_peered":"2023-12-16T21:42:42.472944+0000","last_clean":"2023-12-16T21:42:42.472944+0000","last_became_active":"2023-12-16T21:42:41.365191+0000","last_became_peered":"2023-12-16T21:42:41.365191+0000","last_unstale":"2023-12-16T21:42:42.472944+0000","last_undegraded":"2023-12-16T21:42:42.472944+0000","last_fullsized":"2023-12-16T21:42:42.472944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476214+0000","last_change":"2023-12-16T21:42:40.447186+0000","last_active":"2023-12-16T21:42:42.476214+0000","last_peered":"2023-12-16T21:42:42.476214+0000","last_clean":"2023-12-16T21:42:42.476214+0000","last_became_active":"2023-12-16T21:42:40.446374+0000","last_became_peered":"2023-12-16T21:42:40.446374+0000","last_unstale":"2023-12-16T21:42:42.476214+0000","last_undegraded":"2023-12-16T21:42:42.476214+0000","last_fullsized":"2023-12-16T21:42:42.476214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471782+0000","last_change":"2023-12-16T21:42:41.364750+0000","last_active":"2023-12-16T21:42:42.471782+0000","last_peered":"2023-12-16T21:42:42.471782+0000","last_clean":"2023-12-16T21:42:42.471782+0000","last_became_active":"2023-12-16T21:42:41.363976+0000","last_became_peered":"2023-12-16T21:42:41.363976+0000","last_unstale":"2023-12-16T21:42:42.471782+0000","last_undegraded":"2023-12-16T21:42:42.471782+0000","last_fullsized":"2023-12-16T21:42:42.471782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332140+0000","last_change":"2023-12-16T21:42:40.444481+0000","last_active":"2023-12-16T21:42:42.332140+0000","last_peered":"2023-12-16T21:42:42.332140+0000","last_clean":"2023-12-16T21:42:42.332140+0000","last_became_active":"2023-12-16T21:42:40.443665+0000","last_became_peered":"2023-12-16T21:42:40.443665+0000","last_unstale":"2023-12-16T21:42:42.332140+0000","last_undegraded":"2023-12-16T21:42:42.332140+0000","last_fullsized":"2023-12-16T21:42:42.332140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334181+0000","last_change":"2023-12-16T21:42:41.360039+0000","last_active":"2023-12-16T21:42:42.334181+0000","last_peered":"2023-12-16T21:42:42.334181+0000","last_clean":"2023-12-16T21:42:42.334181+0000","last_became_active":"2023-12-16T21:42:41.359132+0000","last_became_peered":"2023-12-16T21:42:41.359132+0000","last_unstale":"2023-12-16T21:42:42.334181+0000","last_undegraded":"2023-12-16T21:42:42.334181+0000","last_fullsized":"2023-12-16T21:42:42.334181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:52.703815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329301+0000","last_change":"2023-12-16T21:42:40.438910+0000","last_active":"2023-12-16T21:42:42.329301+0000","last_peered":"2023-12-16T21:42:42.329301+0000","last_clean":"2023-12-16T21:42:42.329301+0000","last_became_active":"2023-12-16T21:42:40.438462+0000","last_became_peered":"2023-12-16T21:42:40.438462+0000","last_unstale":"2023-12-16T21:42:42.329301+0000","last_undegraded":"2023-12-16T21:42:42.329301+0000","last_fullsized":"2023-12-16T21:42:42.329301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472184+0000","last_change":"2023-12-16T21:42:41.364881+0000","last_active":"2023-12-16T21:42:42.472184+0000","last_peered":"2023-12-16T21:42:42.472184+0000","last_clean":"2023-12-16T21:42:42.472184+0000","last_became_active":"2023-12-16T21:42:41.363545+0000","last_became_peered":"2023-12-16T21:42:41.363545+0000","last_unstale":"2023-12-16T21:42:42.472184+0000","last_undegraded":"2023-12-16T21:42:42.472184+0000","last_fullsized":"2023-12-16T21:42:42.472184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329671+0000","last_change":"2023-12-16T21:42:40.418011+0000","last_active":"2023-12-16T21:42:42.329671+0000","last_peered":"2023-12-16T21:42:42.329671+0000","last_clean":"2023-12-16T21:42:42.329671+0000","last_became_active":"2023-12-16T21:42:40.417587+0000","last_became_peered":"2023-12-16T21:42:40.417587+0000","last_unstale":"2023-12-16T21:42:42.329671+0000","last_undegraded":"2023-12-16T21:42:42.329671+0000","last_fullsized":"2023-12-16T21:42:42.329671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472662+0000","last_change":"2023-12-16T21:42:41.380236+0000","last_active":"2023-12-16T21:42:42.472662+0000","last_peered":"2023-12-16T21:42:42.472662+0000","last_clean":"2023-12-16T21:42:42.472662+0000","last_became_active":"2023-12-16T21:42:41.379681+0000","last_became_peered":"2023-12-16T21:42:41.379681+0000","last_unstale":"2023-12-16T21:42:42.472662+0000","last_undegraded":"2023-12-16T21:42:42.472662+0000","last_fullsized":"2023-12-16T21:42:42.472662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334158+0000","last_change":"2023-12-16T21:42:40.448985+0000","last_active":"2023-12-16T21:42:42.334158+0000","last_peered":"2023-12-16T21:42:42.334158+0000","last_clean":"2023-12-16T21:42:42.334158+0000","last_became_active":"2023-12-16T21:42:40.443742+0000","last_became_peered":"2023-12-16T21:42:40.443742+0000","last_unstale":"2023-12-16T21:42:42.334158+0000","last_undegraded":"2023-12-16T21:42:42.334158+0000","last_fullsized":"2023-12-16T21:42:42.334158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:53.161999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474831+0000","last_change":"2023-12-16T21:42:41.363951+0000","last_active":"2023-12-16T21:42:42.474831+0000","last_peered":"2023-12-16T21:42:42.474831+0000","last_clean":"2023-12-16T21:42:42.474831+0000","last_became_active":"2023-12-16T21:42:41.363161+0000","last_became_peered":"2023-12-16T21:42:41.363161+0000","last_unstale":"2023-12-16T21:42:42.474831+0000","last_undegraded":"2023-12-16T21:42:42.474831+0000","last_fullsized":"2023-12-16T21:42:42.474831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475088+0000","last_change":"2023-12-16T21:42:41.386385+0000","last_active":"2023-12-16T21:42:42.475088+0000","last_peered":"2023-12-16T21:42:42.475088+0000","last_clean":"2023-12-16T21:42:42.475088+0000","last_became_active":"2023-12-16T21:42:41.385339+0000","last_became_peered":"2023-12-16T21:42:41.385339+0000","last_unstale":"2023-12-16T21:42:42.475088+0000","last_undegraded":"2023-12-16T21:42:42.475088+0000","last_fullsized":"2023-12-16T21:42:42.475088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332799+0000","last_change":"2023-12-16T21:42:40.462751+0000","last_active":"2023-12-16T21:42:42.332799+0000","last_peered":"2023-12-16T21:42:42.332799+0000","last_clean":"2023-12-16T21:42:42.332799+0000","last_became_active":"2023-12-16T21:42:40.461936+0000","last_became_peered":"2023-12-16T21:42:40.461936+0000","last_unstale":"2023-12-16T21:42:42.332799+0000","last_undegraded":"2023-12-16T21:42:42.332799+0000","last_fullsized":"2023-12-16T21:42:42.332799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474683+0000","last_change":"2023-12-16T21:42:41.384715+0000","last_active":"2023-12-16T21:42:42.474683+0000","last_peered":"2023-12-16T21:42:42.474683+0000","last_clean":"2023-12-16T21:42:42.474683+0000","last_became_active":"2023-12-16T21:42:41.383596+0000","last_became_peered":"2023-12-16T21:42:41.383596+0000","last_unstale":"2023-12-16T21:42:42.474683+0000","last_undegraded":"2023-12-16T21:42:42.474683+0000","last_fullsized":"2023-12-16T21:42:42.474683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335044+0000","last_change":"2023-12-16T21:42:40.445218+0000","last_active":"2023-12-16T21:42:42.335044+0000","last_peered":"2023-12-16T21:42:42.335044+0000","last_clean":"2023-12-16T21:42:42.335044+0000","last_became_active":"2023-12-16T21:42:40.443892+0000","last_became_peered":"2023-12-16T21:42:40.443892+0000","last_unstale":"2023-12-16T21:42:42.335044+0000","last_undegraded":"2023-12-16T21:42:42.335044+0000","last_fullsized":"2023-12-16T21:42:42.335044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:20.422855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471754+0000","last_change":"2023-12-16T21:42:41.385409+0000","last_active":"2023-12-16T21:42:42.471754+0000","last_peered":"2023-12-16T21:42:42.471754+0000","last_clean":"2023-12-16T21:42:42.471754+0000","last_became_active":"2023-12-16T21:42:41.384925+0000","last_became_peered":"2023-12-16T21:42:41.384925+0000","last_unstale":"2023-12-16T21:42:42.471754+0000","last_undegraded":"2023-12-16T21:42:42.471754+0000","last_fullsized":"2023-12-16T21:42:42.471754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472993+0000","last_change":"2023-12-16T21:42:40.417361+0000","last_active":"2023-12-16T21:42:42.472993+0000","last_peered":"2023-12-16T21:42:42.472993+0000","last_clean":"2023-12-16T21:42:42.472993+0000","last_became_active":"2023-12-16T21:42:40.416759+0000","last_became_peered":"2023-12-16T21:42:40.416759+0000","last_unstale":"2023-12-16T21:42:42.472993+0000","last_undegraded":"2023-12-16T21:42:42.472993+0000","last_fullsized":"2023-12-16T21:42:42.472993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472885+0000","last_change":"2023-12-16T21:42:41.370637+0000","last_active":"2023-12-16T21:42:42.472885+0000","last_peered":"2023-12-16T21:42:42.472885+0000","last_clean":"2023-12-16T21:42:42.472885+0000","last_became_active":"2023-12-16T21:42:41.369929+0000","last_became_peered":"2023-12-16T21:42:41.369929+0000","last_unstale":"2023-12-16T21:42:42.472885+0000","last_undegraded":"2023-12-16T21:42:42.472885+0000","last_fullsized":"2023-12-16T21:42:42.472885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471919+0000","last_change":"2023-12-16T21:42:40.439054+0000","last_active":"2023-12-16T21:42:42.471919+0000","last_peered":"2023-12-16T21:42:42.471919+0000","last_clean":"2023-12-16T21:42:42.471919+0000","last_became_active":"2023-12-16T21:42:40.438547+0000","last_became_peered":"2023-12-16T21:42:40.438547+0000","last_unstale":"2023-12-16T21:42:42.471919+0000","last_undegraded":"2023-12-16T21:42:42.471919+0000","last_fullsized":"2023-12-16T21:42:42.471919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":802816,"data_stored":660231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2729999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5649999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":208896,"data_stored":69371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.228}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.081}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.113}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2070000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":69281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.619}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":811008,"data_stored":660287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:42:44.782 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-16T21:42:44.782 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-16T21:42:44.782 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-16T21:42:44.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-16T21:42:44.783 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-16T21:42:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.935+0000 7fcbbcfa9640 1 Processor -- start 2023-12-16T21:42:44.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.935+0000 7fcbbcfa9640 1 -- start start 2023-12-16T21:42:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbbcfa9640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb80afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.945 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbbcfa9640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcbb80b0130 0x7fcbb814e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbbcfa9640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb80ad810 con 0x7fcbb814ed30 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbbcfa9640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb806cd00 con 0x7fcbb80b0130 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbbcfa9640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80af480 con 0x7fcbb806c920 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb80afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb5d9b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcbb80b0130 0x7fcbb814e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb659c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb80afbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50354/0 (socket says 172.21.15.46:50354) 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb659c640 1 -- 172.21.15.46:0/1066687069 learned_addr learned my addr 172.21.15.46:0/1066687069 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:44.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb6d9d640 1 --1- 172.21.15.46:0/1066687069 >> v1:172.21.15.161:6789/0 conn(0x7fcbb814ed30 0x7fcbb80b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:54760/0 (socket says 172.21.15.46:54760) 2023-12-16T21:42:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb559a640 1 -- 172.21.15.46:0/1066687069 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001700417 0 0) 0x7fcbb80ad810 con 0x7fcbb814ed30 2023-12-16T21:42:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb559a640 1 -- 172.21.15.46:0/1066687069 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcba0003530 con 0x7fcbb814ed30 2023-12-16T21:42:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb659c640 1 -- 172.21.15.46:0/1066687069 >> v2:172.21.15.161:3300/0 conn(0x7fcbb80b0130 msgr2=0x7fcbb814e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb659c640 1 --2- 172.21.15.46:0/1066687069 >> v2:172.21.15.161:3300/0 conn(0x7fcbb80b0130 0x7fcbb814e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb659c640 1 -- 172.21.15.46:0/1066687069 >> v1:172.21.15.161:6789/0 conn(0x7fcbb814ed30 legacy=0x7fcbb80b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb659c640 1 -- 172.21.15.46:0/1066687069 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80ad210 con 0x7fcbb806c920 2023-12-16T21:42:44.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.939+0000 7fcbb5d9b640 1 --2- 172.21.15.46:0/1066687069 >> v2:172.21.15.161:3300/0 conn(0x7fcbb80b0130 0x7fcbb814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb659c640 1 --2- 172.21.15.46:0/1066687069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb80afbf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009870 tx=0x7fcba801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39b363cf79f29fc server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb559a640 1 -- 172.21.15.46:0/1066687069 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80a7070 con 0x7fcbb806c920 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb559a640 1 -- 172.21.15.46:0/1066687069 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba80ab430 con 0x7fcbb806c920 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb559a640 1 -- 172.21.15.46:0/1066687069 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80ab7c0 con 0x7fcbb806c920 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/1066687069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 msgr2=0x7fcbb80afbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/1066687069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb80afbf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009870 tx=0x7fcba801c540 comp rx=0 tx=0).stop 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/1066687069 shutdown_connections 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/1066687069 >> v2:172.21.15.161:3300/0 conn(0x7fcbb80b0130 0x7fcbb814e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/1066687069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb80afbf0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/1066687069 >> 172.21.15.46:0/1066687069 conn(0x7fcbb80b48e0 msgr2=0x7fcbb80b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/1066687069 shutdown_connections 2023-12-16T21:42:44.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/1066687069 wait complete. 2023-12-16T21:42:44.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 Processor -- start 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- start start 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb809d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb80b0130 0x7fcbb8099dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb814ed30 0x7fcbb809a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb809b940 con 0x7fcbb814ed30 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb809c980 con 0x7fcbb80b0130 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbbcfa9640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb809cc60 con 0x7fcbb806c920 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb814ed30 0x7fcbb809a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb6d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb814ed30 0x7fcbb809a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44936/0 (socket says 172.21.15.46:44936) 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb6d9d640 1 -- 172.21.15.46:0/3010013361 learned_addr learned my addr 172.21.15.46:0/3010013361 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb659c640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb809d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.943+0000 7fcbb5d9b640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb80b0130 0x7fcbb8099dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbb6d9d640 1 -- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 msgr2=0x7fcbb809d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbb6d9d640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb809d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbb6d9d640 1 -- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb80b0130 msgr2=0x7fcbb8099dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbb6d9d640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb80b0130 0x7fcbb8099dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbb6d9d640 1 -- 172.21.15.46:0/3010013361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac009420 con 0x7fcbb814ed30 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbb6d9d640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb814ed30 0x7fcbb809a310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000c770 tx=0x7fcbb000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcba6ffd640 1 -- 172.21.15.46:0/3010013361 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000ce80 con 0x7fcbb814ed30 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcba6ffd640 1 -- 172.21.15.46:0/3010013361 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbb000a640 con 0x7fcbb814ed30 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba80094d0 con 0x7fcbb814ed30 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcba6ffd640 1 -- 172.21.15.46:0/3010013361 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbb000a9f0 con 0x7fcbb814ed30 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb809b5a0 con 0x7fcbb814ed30 2023-12-16T21:42:44.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb80679e0 con 0x7fcbb814ed30 2023-12-16T21:42:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcba6ffd640 1 -- 172.21.15.46:0/3010013361 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fcbb000ab50 con 0x7fcbb814ed30 2023-12-16T21:42:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcba6ffd640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb98066470 0x7fcb98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcba6ffd640 1 -- 172.21.15.46:0/3010013361 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbb0083500 con 0x7fcbb814ed30 2023-12-16T21:42:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.947+0000 7fcbb659c640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb98066470 0x7fcb98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:44.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.951+0000 7fcbb659c640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb98066470 0x7fcb98068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcba80af000 tx=0x7fcba8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:44.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:44.955+0000 7fcba6ffd640 1 -- 172.21.15.46:0/3010013361 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fcbb0050aa0 con 0x7fcbb814ed30 2023-12-16T21:42:45.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.143+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fcbb80688f0 con 0x7fcbb814ed30 2023-12-16T21:42:45.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.147+0000 7fcba6ffd640 1 -- 172.21.15.46:0/3010013361 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164438 (secure 0 0 0) 0x7fcbb00540c0 con 0x7fcbb814ed30 2023-12-16T21:42:45.158 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb98066470 msgr2=0x7fcb98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb98066470 0x7fcb98068930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcba80af000 tx=0x7fcba8008040 comp rx=0 tx=0).stop 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb814ed30 msgr2=0x7fcbb809a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb814ed30 0x7fcbb809a310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcbb000c770 tx=0x7fcbb000cc30 comp rx=0 tx=0).stop 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 shutdown_connections 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb98066470 0x7fcb98068930 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbb814ed30 0x7fcbb809a310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcbb80b0130 0x7fcbb8099dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 --2- 172.21.15.46:0/3010013361 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcbb806c920 0x7fcbb809d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 >> 172.21.15.46:0/3010013361 conn(0x7fcbb80b48e0 msgr2=0x7fcbb8071fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 shutdown_connections 2023-12-16T21:42:45.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.151+0000 7fcbbcfa9640 1 -- 172.21.15.46:0/3010013361 wait complete. 2023-12-16T21:42:45.175 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6832","nonce":4278008418},{"type":"v1","addr":"172.21.15.161:6833","nonce":4278008418}]},"active_addr":"172.21.15.161:6833/4278008418","active_change":"2023-12-16T21:42:20.333318+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":925745676}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":932360696}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":2310830289}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.161:0","nonce":824263277}]}]} 2023-12-16T21:42:45.178 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-16T21:42:45.179 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-16T21:42:45.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:45.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.291+0000 7f4742edd640 1 Processor -- start 2023-12-16T21:42:45.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4742edd640 1 -- start start 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4742edd640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f473c1402d0 0x7f473c140700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4742edd640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473c0b4690 con 0x7f473c0bc830 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4742edd640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473c0b8740 con 0x7f473c149ef0 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4742edd640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f473c0bcc10 con 0x7f473c1402d0 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4740c79640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f473c0bc830 0x7f473c0b3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43010/0 (socket says 172.21.15.46:43010) 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4740c79640 1 -- 172.21.15.46:0/1351453091 learned_addr learned my addr 172.21.15.46:0/1351453091 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f474147a640 1 --2- 172.21.15.46:0/1351453091 >> v2:172.21.15.161:3300/0 conn(0x7f473c1402d0 0x7f473c140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014578316 0 0) 0x7f473c0b4690 con 0x7f473c0bc830 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f472c003530 con 0x7f473c0bc830 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1719785999 0 0) 0x7f472c003530 con 0x7f473c0bc830 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 >> v2:172.21.15.161:3300/0 conn(0x7f473c1402d0 msgr2=0x7f473c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 --2- 172.21.15.46:0/1351453091 >> v2:172.21.15.161:3300/0 conn(0x7f473c1402d0 0x7f473c140700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 >> v1:172.21.15.161:6789/0 conn(0x7f473c149ef0 legacy=0x7f473c14a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0b48e0 con 0x7f473c0bc830 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f4730001680 con 0x7f473c0bc830 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 >> v1:172.21.15.46:6789/0 conn(0x7f473c0bc830 legacy=0x7f473c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f472c003980 0x7f472c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f472c0047d0 0x7f472c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f472c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0bcc10 con 0x7f472c005190 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f472c00c130 con 0x7f472c0047d0 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f472c00c2f0 con 0x7f472c003980 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f474147a640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f472c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f473bfff640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f472c0047d0 0x7f472c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f4740c79640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f472c003980 0x7f472c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f474147a640 1 -- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f472c003980 msgr2=0x7f472c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f474147a640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f472c003980 0x7f472c003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f474147a640 1 -- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f472c0047d0 msgr2=0x7f472c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f474147a640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f472c0047d0 0x7f472c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.295+0000 7f474147a640 1 -- 172.21.15.46:0/1351453091 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c003530 con 0x7f472c005190 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f474147a640 1 -- 172.21.15.46:0/1351453091 reap_dead start 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f474147a640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f472c00b990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4734009fc0 tx=0x7f473401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81be942306e70bbf server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47340a5cb0 con 0x7f472c005190 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473400f040 con 0x7f472c005190 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f473b7fe640 1 -- 172.21.15.46:0/1351453091 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4734003be0 con 0x7f472c005190 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- 172.21.15.46:0/1351453091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 msgr2=0x7f472c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f472c00b990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4734009fc0 tx=0x7f473401cca0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- 172.21.15.46:0/1351453091 shutdown_connections 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 --2- 172.21.15.46:0/1351453091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f472c00b990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- 172.21.15.46:0/1351453091 >> 172.21.15.46:0/1351453091 conn(0x7f473c072340 msgr2=0x7f473c0bbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- 172.21.15.46:0/1351453091 shutdown_connections 2023-12-16T21:42:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- 172.21.15.46:0/1351453091 wait complete. 2023-12-16T21:42:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 Processor -- start 2023-12-16T21:42:45.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- start start 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f473c147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c0bb270 0x7f473c0bbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f473c0bc0a0 0x7f473c1340e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c141c20 con 0x7f472c005190 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0bade0 con 0x7f473c0bb270 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4742edd640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f473c0ba4f0 con 0x7f473c0bc0a0 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f473c147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f473c147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44952/0 (socket says 172.21.15.46:44952) 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 -- 172.21.15.46:0/979425769 learned_addr learned my addr 172.21.15.46:0/979425769 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f473bfff640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c0bb270 0x7f473c0bbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f474147a640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f473c0bc0a0 0x7f473c1340e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 -- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f473c0bc0a0 msgr2=0x7f473c1340e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f473c0bc0a0 0x7f473c1340e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 -- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c0bb270 msgr2=0x7f473c0bbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c0bb270 0x7f473c0bbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 -- 172.21.15.46:0/979425769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0ba790 con 0x7f472c005190 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f473bfff640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c0bb270 0x7f473c0bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.299+0000 7f4740c79640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f473c147880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4730010b80 tx=0x7f473000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f47397fa640 1 -- 172.21.15.46:0/979425769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473000ed80 con 0x7f472c005190 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f47397fa640 1 -- 172.21.15.46:0/979425769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47300143a0 con 0x7f472c005190 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f47397fa640 1 -- 172.21.15.46:0/979425769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730014730 con 0x7f472c005190 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f473c0baa70 con 0x7f472c005190 2023-12-16T21:42:45.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f473c147dc0 con 0x7f472c005190 2023-12-16T21:42:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f473c067e60 con 0x7f472c005190 2023-12-16T21:42:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f47397fa640 1 -- 172.21.15.46:0/979425769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f4730014890 con 0x7f472c005190 2023-12-16T21:42:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f47397fa640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f47200664a0 0x7f4720068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f47397fa640 1 -- 172.21.15.46:0/979425769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4730086a90 con 0x7f472c005190 2023-12-16T21:42:45.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f473bfff640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f47200664a0 0x7f4720068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.303+0000 7f473bfff640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f47200664a0 0x7f4720068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4728000c10 tx=0x7f4728009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.307+0000 7f47397fa640 1 -- 172.21.15.46:0/979425769 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f4730053f10 con 0x7f472c005190 2023-12-16T21:42:45.336 INFO:tasks.ceph.mds.b.smithi161.stderr:2023-12-16T21:42:45.328+0000 7f3edb8f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-16T21:42:45.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.463+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f473c0688f0 con 0x7f472c005190 2023-12-16T21:42:45.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f47397fa640 1 -- 172.21.15.46:0/979425769 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f473001b020 con 0x7f472c005190 2023-12-16T21:42:45.474 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:45.474 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:45.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f47200664a0 msgr2=0x7f4720068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f47200664a0 0x7f4720068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4728000c10 tx=0x7f4728009040 comp rx=0 tx=0).stop 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 msgr2=0x7f473c147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f473c147880 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4730010b80 tx=0x7f473000e4d0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 shutdown_connections 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f47200664a0 0x7f4720068960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f473c0bc0a0 0x7f473c1340e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f473c0bb270 0x7f473c0bbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 --2- 172.21.15.46:0/979425769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f472c005190 0x7f473c147880 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.467+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 >> 172.21.15.46:0/979425769 conn(0x7f473c072340 msgr2=0x7f473c144730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.471+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 shutdown_connections 2023-12-16T21:42:45.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.471+0000 7f4742edd640 1 -- 172.21.15.46:0/979425769 wait complete. 2023-12-16T21:42:45.493 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-16T21:42:45.493 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.607+0000 7f2cdf199640 1 Processor -- start 2023-12-16T21:42:45.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.607+0000 7f2cdf199640 1 -- start start 2023-12-16T21:42:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdf199640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 0x7f2cd8156a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdf199640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd80ad810 con 0x7f2cd806c920 2023-12-16T21:42:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdf199640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd806cd00 con 0x7f2cd814e7f0 2023-12-16T21:42:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdf199640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd80af480 con 0x7f2cd8154680 2023-12-16T21:42:45.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdcf35640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2cd806c920 0x7f2cd80afbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43016/0 (socket says 172.21.15.46:43016) 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdcf35640 1 -- 172.21.15.46:0/267383691 learned_addr learned my addr 172.21.15.46:0/267383691 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdd736640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 0x7f2cd8156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1380505535 0 0) 0x7f2cd80ad810 con 0x7f2cd806c920 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cb8003530 con 0x7f2cd806c920 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 391235466 0 0) 0x7f2cb8003530 con 0x7f2cd806c920 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 msgr2=0x7f2cd8156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 0x7f2cd8156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 >> v1:172.21.15.161:6789/0 conn(0x7f2cd814e7f0 legacy=0x7f2cd814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80ad210 con 0x7f2cd806c920 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f2cc8001680 con 0x7f2cd806c920 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 >> v1:172.21.15.46:6789/0 conn(0x7f2cd806c920 legacy=0x7f2cd80afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80039b0 0x7f2cb8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb80047d0 0x7f2cb8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 0x7f2cb8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd80af480 con 0x7f2cb80047d0 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb800ba70 con 0x7f2cb80039b0 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb800bc30 con 0x7f2cd8154680 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd7fff640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb80047d0 0x7f2cb8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdcf35640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80039b0 0x7f2cb8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cdd736640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 0x7f2cb8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd7fff640 1 -- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 msgr2=0x7f2cb8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd7fff640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd8154680 0x7f2cb8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd7fff640 1 -- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80039b0 msgr2=0x7f2cb8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd7fff640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80039b0 0x7f2cb8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd7fff640 1 -- 172.21.15.46:0/267383691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb8003530 con 0x7f2cb80047d0 2023-12-16T21:42:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd7fff640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb80047d0 0x7f2cb8004c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0001e70 tx=0x7f2cc001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8f7575c0946891 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0003b80 con 0x7f2cb80047d0 2023-12-16T21:42:45.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc0003ce0 con 0x7f2cb80047d0 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.611+0000 7f2cd77fe640 1 -- 172.21.15.46:0/267383691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00052c0 con 0x7f2cb80047d0 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- 172.21.15.46:0/267383691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb80047d0 msgr2=0x7f2cb8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 --2- 172.21.15.46:0/267383691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cb80047d0 0x7f2cb8004c50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0001e70 tx=0x7f2cc001c760 comp rx=0 tx=0).stop 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 -- 172.21.15.46:0/267383691 reap_dead start 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- 172.21.15.46:0/267383691 shutdown_connections 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- 172.21.15.46:0/267383691 >> 172.21.15.46:0/267383691 conn(0x7f2cd80b48e0 msgr2=0x7f2cd80b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- 172.21.15.46:0/267383691 shutdown_connections 2023-12-16T21:42:45.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- 172.21.15.46:0/267383691 wait complete. 2023-12-16T21:42:45.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 Processor -- start 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- start start 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80047d0 0x7f2cd806f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd806f930 0x7f2cd806fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd81e34b0 0x7f2cd81e3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81da800 con 0x7f2cd81e34b0 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81da370 con 0x7f2cb80047d0 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd81d9d20 con 0x7f2cd806f930 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd81e34b0 0x7f2cd81e3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd81e34b0 0x7f2cd81e3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44974/0 (socket says 172.21.15.46:44974) 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 -- 172.21.15.46:0/3074550166 learned_addr learned my addr 172.21.15.46:0/3074550166 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cd7fff640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd806f930 0x7f2cd806fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdcf35640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80047d0 0x7f2cd806f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 -- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd806f930 msgr2=0x7f2cd806fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.624 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd806f930 0x7f2cd806fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 -- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80047d0 msgr2=0x7f2cd806f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80047d0 0x7f2cd806f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 -- 172.21.15.46:0/3074550166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd81da000 con 0x7f2cd81e34b0 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdd736640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd81e34b0 0x7f2cd81e3930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80b57a0 tx=0x7f2cd0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cd57fa640 1 -- 172.21.15.46:0/3074550166 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd0004500 con 0x7f2cd81e34b0 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cd57fa640 1 -- 172.21.15.46:0/3074550166 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd0003620 con 0x7f2cd81e34b0 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd814d5a0 con 0x7f2cd81e34b0 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cd57fa640 1 -- 172.21.15.46:0/3074550166 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd00039d0 con 0x7f2cd81e34b0 2023-12-16T21:42:45.625 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.615+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd809dcc0 con 0x7f2cd81e34b0 2023-12-16T21:42:45.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.619+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd80679e0 con 0x7f2cd81e34b0 2023-12-16T21:42:45.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.619+0000 7f2cd57fa640 1 -- 172.21.15.46:0/3074550166 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2cd0003b30 con 0x7f2cd81e34b0 2023-12-16T21:42:45.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.619+0000 7f2cd57fa640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cbc066470 0x7f2cbc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.619+0000 7f2cd57fa640 1 -- 172.21.15.46:0/3074550166 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cd00840f0 con 0x7f2cd81e34b0 2023-12-16T21:42:45.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.619+0000 7f2cdcf35640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cbc066470 0x7f2cbc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.626 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.619+0000 7f2cdcf35640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cbc066470 0x7f2cbc068930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8001210 tx=0x7f2cc80014c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.623+0000 7f2cd57fa640 1 -- 172.21.15.46:0/3074550166 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2cd0051580 con 0x7f2cd81e34b0 2023-12-16T21:42:45.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.771+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cd81da190 con 0x7f2cd81e34b0 2023-12-16T21:42:45.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cd57fa640 1 -- 172.21.15.46:0/3074550166 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f2cd0054ba0 con 0x7f2cd81e34b0 2023-12-16T21:42:45.782 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:45.782 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cbc066470 msgr2=0x7f2cbc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cbc066470 0x7f2cbc068930 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2cc8001210 tx=0x7f2cc80014c0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd81e34b0 msgr2=0x7f2cd81e3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd81e34b0 0x7f2cd81e3930 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80b57a0 tx=0x7f2cd0011920 comp rx=0 tx=0).stop 2023-12-16T21:42:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 shutdown_connections 2023-12-16T21:42:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cbc066470 0x7f2cbc068930 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cd81e34b0 0x7f2cd81e3930 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cd806f930 0x7f2cd806fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 --2- 172.21.15.46:0/3074550166 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cb80047d0 0x7f2cd806f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 >> 172.21.15.46:0/3074550166 conn(0x7f2cd80b48e0 msgr2=0x7f2cd8066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 shutdown_connections 2023-12-16T21:42:45.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.775+0000 7f2cdf199640 1 -- 172.21.15.46:0/3074550166 wait complete. 2023-12-16T21:42:45.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-16T21:42:45.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-16T21:42:45.801 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-16T21:42:45.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-16T21:42:45.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-16T21:42:45.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-16T21:42:45.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-16T21:42:45.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-16T21:42:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.971+0000 7f2f33fff640 1 Processor -- start 2023-12-16T21:42:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.971+0000 7f2f33fff640 1 -- start start 2023-12-16T21:42:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f33fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071f80 0x7f2f3414e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f33fff640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f34142950 con 0x7f2f340b05d0 2023-12-16T21:42:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f33fff640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f341434f0 con 0x7f2f34071b40 2023-12-16T21:42:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f33fff640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f340af660 con 0x7f2f34071f80 2023-12-16T21:42:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f91116da640 1 Processor -- start 2023-12-16T21:42:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f91116da640 1 -- start start 2023-12-16T21:42:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f32ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2f340b05d0 0x7f2f34142de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43028/0 (socket says 172.21.15.46:43028) 2023-12-16T21:42:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f337fe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071f80 0x7f2f3414e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f32ffd640 1 -- 172.21.15.46:0/3402753063 learned_addr learned my addr 172.21.15.46:0/3402753063 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f31ffb640 1 -- 172.21.15.46:0/3402753063 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500046954 0 0) 0x7f2f34142950 con 0x7f2f340b05d0 2023-12-16T21:42:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f31ffb640 1 -- 172.21.15.46:0/3402753063 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f1c003450 con 0x7f2f340b05d0 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f91116da640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9104036d60 0x7f9104034360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f91116da640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91040348a0 0x7f9104035850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f91116da640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91040d18b0 con 0x7f9104036980 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f91116da640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9104034d30 con 0x7f9104036d60 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f91116da640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9104035050 con 0x7f91040348a0 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910a7fc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9104036d60 0x7f9104034360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910affd640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f9104036980 0x7f91040d0e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:54794/0 (socket says 172.21.15.46:54794) 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910affd640 1 -- 172.21.15.46:0/1881018690 learned_addr learned my addr 172.21.15.46:0/1881018690 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910bfff640 1 --2- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91040348a0 0x7f9104035850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910a7fc640 1 -- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91040348a0 msgr2=0x7f9104035850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910a7fc640 1 --2- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91040348a0 0x7f9104035850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910a7fc640 1 -- 172.21.15.46:0/1881018690 >> v1:172.21.15.161:6789/0 conn(0x7f9104036980 legacy=0x7f91040d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f910a7fc640 1 -- 172.21.15.46:0/1881018690 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104035330 con 0x7f9104036d60 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7ff8e359e640 1 Processor -- start 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7ff8e359e640 1 -- start start 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7ff8e359e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc0c2f00 0x7ff8dc0c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7ff8e359e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff8dc039fc0 0x7ff8dc0c9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7ff8e359e640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8dc02a420 con 0x7ff8dc038e50 2023-12-16T21:42:45.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7ff8e359e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8dc02a1b0 con 0x7ff8dc039fc0 2023-12-16T21:42:45.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7ff8e359e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc039230 con 0x7ff8dc0c2f00 2023-12-16T21:42:45.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e259c640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7ff8dc038e50 0x7ff8dc0c27f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:54798/0 (socket says 172.21.15.46:54798) 2023-12-16T21:42:45.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e259c640 1 -- 172.21.15.46:0/3524783978 learned_addr learned my addr 172.21.15.46:0/3524783978 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e1d9b640 1 --2- 172.21.15.46:0/3524783978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc0c2f00 0x7ff8dc0c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f337fe640 1 -- 172.21.15.46:0/3402753063 >> v1:172.21.15.161:6789/0 conn(0x7f2f34071b40 legacy=0x7f2f3414ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f337fe640 1 -- 172.21.15.46:0/3402753063 >> v1:172.21.15.46:6789/0 conn(0x7f2f340b05d0 legacy=0x7f2f34142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.975+0000 7f2f337fe640 1 -- 172.21.15.46:0/3402753063 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f340b09b0 con 0x7f2f34071f80 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e2d9d640 1 --2- 172.21.15.46:0/3524783978 >> v2:172.21.15.46:3300/0 conn(0x7ff8dc039fc0 0x7ff8dc0c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e159a640 1 -- 172.21.15.46:0/3524783978 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541321104 0 0) 0x7ff8dc02a420 con 0x7ff8dc038e50 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e159a640 1 -- 172.21.15.46:0/3524783978 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8cc003530 con 0x7ff8dc038e50 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e2d9d640 1 -- 172.21.15.46:0/3524783978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc0c2f00 msgr2=0x7ff8dc0c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e2d9d640 1 --2- 172.21.15.46:0/3524783978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc0c2f00 0x7ff8dc0c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e2d9d640 1 -- 172.21.15.46:0/3524783978 >> v1:172.21.15.161:6789/0 conn(0x7ff8dc038e50 legacy=0x7ff8dc0c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff8e2d9d640 1 -- 172.21.15.46:0/3524783978 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc037160 con 0x7ff8dc039fc0 2023-12-16T21:42:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f337fe640 1 --2- 172.21.15.46:0/3402753063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071f80 0x7f2f3414e570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0099c0 tx=0x7f2f2c01cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b32493197fe1c01 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb759e640 1 Processor -- start 2023-12-16T21:42:45.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f31ffb640 1 -- 172.21.15.46:0/3402753063 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0a5b80 con 0x7f2f34071f80 2023-12-16T21:42:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf89236640 1 Processor -- start 2023-12-16T21:42:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf89236640 1 -- start start 2023-12-16T21:42:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f31ffb640 1 -- 172.21.15.46:0/3402753063 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2c0a5ce0 con 0x7f2f34071f80 2023-12-16T21:42:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f31ffb640 1 -- 172.21.15.46:0/3402753063 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c0ad660 con 0x7f2f34071f80 2023-12-16T21:42:45.988 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f910a7fc640 1 --2- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9104036d60 0x7f9104034360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91000097c0 tx=0x7f910001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe7e456a2e096778 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb759e640 1 -- start start 2023-12-16T21:42:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f9109ffb640 1 -- 172.21.15.46:0/1881018690 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000ab660 con 0x7f9104036d60 2023-12-16T21:42:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f9109ffb640 1 -- 172.21.15.46:0/1881018690 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91000ab7c0 con 0x7f9104036d60 2023-12-16T21:42:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f9109ffb640 1 -- 172.21.15.46:0/1881018690 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000abb50 con 0x7f9104036d60 2023-12-16T21:42:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f0cfb311640 1 Processor -- start 2023-12-16T21:42:45.989 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f0cfb311640 1 -- start start 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f33fff640 1 -- 172.21.15.46:0/3402753063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071f80 msgr2=0x7f2f3414e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f33fff640 1 --2- 172.21.15.46:0/3402753063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071f80 0x7f2f3414e570 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c0099c0 tx=0x7f2f2c01cbc0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb759e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb80b0540 0x7fccb8071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb759e640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fccb8072080 0x7fccb8146e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb759e640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb8142950 con 0x7fccb80b87c0 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb759e640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb81434f0 con 0x7fccb8072080 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb759e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb80af660 con 0x7fccb80b0540 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb659c640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fccb80b87c0 0x7fccb8142de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:54804/0 (socket says 172.21.15.46:54804) 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb659c640 1 -- 172.21.15.46:0/3291957619 learned_addr learned my addr 172.21.15.46:0/3291957619 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb5d9b640 1 --2- 172.21.15.46:0/3291957619 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb80b0540 0x7fccb8071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f33fff640 1 -- 172.21.15.46:0/3402753063 shutdown_connections 2023-12-16T21:42:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f33fff640 1 --2- 172.21.15.46:0/3402753063 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071f80 0x7f2f3414e570 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f33fff640 1 -- 172.21.15.46:0/3402753063 >> 172.21.15.46:0/3402753063 conn(0x7f2f340b7ca0 msgr2=0x7f2f340b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb559a640 1 -- 172.21.15.46:0/3291957619 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959579433 0 0) 0x7fccb8142950 con 0x7fccb80b87c0 2023-12-16T21:42:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb559a640 1 -- 172.21.15.46:0/3291957619 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcca4003530 con 0x7fccb80b87c0 2023-12-16T21:42:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb5d9b640 1 -- 172.21.15.46:0/3291957619 >> v2:172.21.15.161:3300/0 conn(0x7fccb8072080 msgr2=0x7fccb8146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb5d9b640 1 --2- 172.21.15.46:0/3291957619 >> v2:172.21.15.161:3300/0 conn(0x7fccb8072080 0x7fccb8146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb5d9b640 1 -- 172.21.15.46:0/3291957619 >> v1:172.21.15.161:6789/0 conn(0x7fccb80b87c0 legacy=0x7fccb8142de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f33fff640 1 -- 172.21.15.46:0/3402753063 shutdown_connections 2023-12-16T21:42:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 -- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9104036d60 msgr2=0x7f9104034360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 --2- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9104036d60 0x7f9104034360 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91000097c0 tx=0x7f910001c480 comp rx=0 tx=0).stop 2023-12-16T21:42:45.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9baedf640 1 Processor -- start 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9baedf640 1 -- start start 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9baedf640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff9b40b4f30 0x7ff9b40b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9baedf640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40af700 con 0x7ff9b40aec40 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9baedf640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b40b1e50 con 0x7ff9b40b4300 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9baedf640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b40b5310 con 0x7ff9b40b4f30 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf89236640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf8406c900 0x7fcf840b4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf89236640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcf840afbc0 0x7fcf84070fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf89236640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf840b3ae0 con 0x7fcf840b4d40 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf89236640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf840701c0 con 0x7fcf840afbc0 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf89236640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84071500 con 0x7fcf8406c900 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf8359e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcf840b4d40 0x7fcf841402b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43036/0 (socket says 172.21.15.46:43036) 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf8359e640 1 -- 172.21.15.46:0/1040353865 learned_addr learned my addr 172.21.15.46:0/1040353865 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513488991 0 0) 0x7fcf840b3ae0 con 0x7fcf840b4d40 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf6c003530 con 0x7fcf840b4d40 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 -- 172.21.15.46:0/1881018690 shutdown_connections 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 --2- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f91040348a0 0x7f9104035850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 --2- 172.21.15.46:0/1881018690 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9104036d60 0x7f9104034360 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.994 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 -- 172.21.15.46:0/1881018690 >> 172.21.15.46:0/1881018690 conn(0x7f910403a800 msgr2=0x7f910403ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9b947c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff9b40aec40 0x7ff9b4071ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43044/0 (socket says 172.21.15.46:43044) 2023-12-16T21:42:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9b947c640 1 -- 172.21.15.46:0/3565742914 learned_addr learned my addr 172.21.15.46:0/3565742914 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 -- 172.21.15.46:0/1881018690 shutdown_connections 2023-12-16T21:42:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf8259c640 1 --2- 172.21.15.46:0/1040353865 >> v2:172.21.15.161:3300/0 conn(0x7fcf840afbc0 0x7fcf84070fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.995 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf8406c900 0x7fcf840b4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7ff9b8c7b640 1 --2- 172.21.15.46:0/3565742914 >> v2:172.21.15.161:3300/0 conn(0x7ff9b40b4f30 0x7ff9b40b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b37fe640 1 -- 172.21.15.46:0/3565742914 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493224470 0 0) 0x7ff9b40af700 con 0x7ff9b40aec40 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b37fe640 1 -- 172.21.15.46:0/3565742914 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff99c003530 con 0x7ff9b40aec40 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1040394871 0 0) 0x7fcf6c003530 con 0x7fcf840b4d40 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf8406c900 msgr2=0x7fcf840b4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf8406c900 0x7fcf840b4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 >> v2:172.21.15.161:3300/0 conn(0x7fcf840afbc0 msgr2=0x7fcf84070fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 --2- 172.21.15.46:0/1040353865 >> v2:172.21.15.161:3300/0 conn(0x7fcf840afbc0 0x7fcf84070fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:45.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8406cce0 con 0x7fcf840b4d40 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fcf78001680 con 0x7fcf840b4d40 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 >> v1:172.21.15.46:6789/0 conn(0x7fcf840b4d40 legacy=0x7fcf841402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf6c003980 0x7fcf6c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf6c0047d0 0x7fcf6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6c005190 0x7fcf6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf840701c0 con 0x7fcf6c003980 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf84071500 con 0x7fcf6c0047d0 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c00bed0 con 0x7fcf6c005190 2023-12-16T21:42:45.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e2d9d640 1 --2- 172.21.15.46:0/3524783978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc039fc0 0x7ff8dc0c9120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8e402cef0 tx=0x7ff8e4098cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3e4d576b987d31 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc0cc73640 1 Processor -- start 2023-12-16T21:42:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf8359e640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6c005190 0x7fcf6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e159a640 1 -- 172.21.15.46:0/3524783978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4121bf0 con 0x7ff8dc039fc0 2023-12-16T21:42:45.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf6c003980 0x7fcf6c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc0cc73640 1 -- start start 2023-12-16T21:42:45.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf8259c640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf6c0047d0 0x7fcf6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:45.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f91116da640 1 -- 172.21.15.46:0/1881018690 wait complete. 2023-12-16T21:42:45.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 Processor -- start 2023-12-16T21:42:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f2f33fff640 1 -- 172.21.15.46:0/3402753063 wait complete. 2023-12-16T21:42:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 Processor -- start 2023-12-16T21:42:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 -- start start 2023-12-16T21:42:46.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e159a640 1 -- 172.21.15.46:0/3524783978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e4125070 con 0x7ff8dc039fc0 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e359e640 1 -- 172.21.15.46:0/3524783978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc039fc0 msgr2=0x7ff8dc0c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e159a640 1 -- 172.21.15.46:0/3524783978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4121bf0 con 0x7ff8dc039fc0 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e359e640 1 --2- 172.21.15.46:0/3524783978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc039fc0 0x7ff8dc0c9120 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8e402cef0 tx=0x7ff8e4098cd0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071b40 0x7f2f34071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f34071f80 0x7f2f341335c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f340b05d0 0x7f2f3413cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f34138cb0 con 0x7f2f340b05d0 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f34133b00 con 0x7f2f34071f80 2023-12-16T21:42:46.001 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f33fff640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3413ec70 con 0x7f2f34071b40 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 -- start start 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f0cfb311640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec03ab00 0x7f0cec03af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f0cfb311640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cec0374d0 0x7f0cec037940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f0cfb311640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cec039320 con 0x7f0cec038810 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f0cfb311640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cec038bf0 con 0x7f0cec0374d0 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7f0cfb311640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0395c0 con 0x7f0cec03ab00 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f0cf9b0e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec03ab00 0x7f0cec03af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f0cf9b0e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec03ab00 0x7f0cec03af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50470/0 (socket says 172.21.15.46:50470) 2023-12-16T21:42:46.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f0cf9b0e640 1 -- 172.21.15.46:0/2120562414 learned_addr learned my addr 172.21.15.46:0/2120562414 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.979+0000 7fccb5d9b640 1 -- 172.21.15.46:0/3291957619 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb80b19b0 con 0x7fccb80b0540 2023-12-16T21:42:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b8c7b640 1 -- 172.21.15.46:0/3565742914 >> v1:172.21.15.161:6789/0 conn(0x7ff9b40b4300 legacy=0x7ff9b4142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b8c7b640 1 -- 172.21.15.46:0/3565742914 >> v1:172.21.15.46:6789/0 conn(0x7ff9b40aec40 legacy=0x7ff9b4071ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b8c7b640 1 -- 172.21.15.46:0/3565742914 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40af960 con 0x7ff9b40b4f30 2023-12-16T21:42:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc0cc73640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7efc0003a720 0x7efc0003aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc0cc73640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efc000214a0 0x7efc000218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc0cc73640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc000cb3c0 con 0x7efc0000c330 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc0cc73640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0001fd40 con 0x7efc0003a720 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc0cc73640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc0003b0e0 con 0x7efc000214a0 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071b40 0x7f2f34071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071b40 0x7f2f34071120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50492/0 (socket says 172.21.15.46:50492) 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 -- 172.21.15.46:0/2687436700 learned_addr learned my addr 172.21.15.46:0/2687436700 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f0cf9b0e640 1 -- 172.21.15.46:0/2120562414 >> v2:172.21.15.46:3300/0 conn(0x7f0cec0374d0 msgr2=0x7f0cec037940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f0cf9b0e640 1 --2- 172.21.15.46:0/2120562414 >> v2:172.21.15.46:3300/0 conn(0x7f0cec0374d0 0x7f0cec037940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.004 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f0cf9b0e640 1 -- 172.21.15.46:0/2120562414 >> v1:172.21.15.46:6789/0 conn(0x7f0cec038810 legacy=0x7f0cec0cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fccb5d9b640 1 --2- 172.21.15.46:0/3291957619 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb80b0540 0x7fccb8071b40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009d90 tx=0x7fccb001c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ad37dae9a7455c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91040348a0 0x7f91040c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9104036980 0x7f91040c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9104036d60 0x7f91040c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104169e30 con 0x7f9104036d60 2023-12-16T21:42:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9104169770 con 0x7f91040348a0 2023-12-16T21:42:46.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f91116da640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9104169a50 con 0x7f9104036980 2023-12-16T21:42:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc07fff640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7efc000214a0 0x7efc000218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc077fe640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7efc0000c330 0x7efc000ca550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:54818/0 (socket says 172.21.15.46:54818) 2023-12-16T21:42:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc077fe640 1 -- 172.21.15.46:0/3076925267 learned_addr learned my addr 172.21.15.46:0/3076925267 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fccb559a640 1 -- 172.21.15.46:0/3291957619 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a5880 con 0x7fccb80b0540 2023-12-16T21:42:46.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fccb559a640 1 -- 172.21.15.46:0/3291957619 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb00a59e0 con 0x7fccb80b0540 2023-12-16T21:42:46.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fccb559a640 1 -- 172.21.15.46:0/3291957619 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a5d70 con 0x7fccb80b0540 2023-12-16T21:42:46.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b8c7b640 1 --2- 172.21.15.46:0/3565742914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 0x7ff9b40b09f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80087f0 tx=0x7ff9a801e8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a80df42bb803f9e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f0cf9b0e640 1 -- 172.21.15.46:0/2120562414 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cec0398a0 con 0x7f0cec03ab00 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 -- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f34071f80 msgr2=0x7f2f341335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2f34071f80 0x7f2f341335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 -- 172.21.15.46:0/2687436700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f340b05d0 msgr2=0x7f2f3413cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f340b05d0 0x7f2f3413cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 -- 172.21.15.46:0/2687436700 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c0095f0 con 0x7f2f34071b40 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f910affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91040348a0 0x7f91040c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f910affd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91040348a0 0x7f91040c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:35456/0 (socket says 172.21.15.46:35456) 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f910affd640 1 -- 172.21.15.46:0/2100774795 learned_addr learned my addr 172.21.15.46:0/2100774795 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b37fe640 1 -- 172.21.15.46:0/3565742914 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80a79c0 con 0x7ff9b40b4f30 2023-12-16T21:42:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f910a7fc640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9104036980 0x7f91040c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc07fff640 1 -- 172.21.15.46:0/3076925267 >> v2:172.21.15.46:3300/0 conn(0x7efc0003a720 msgr2=0x7efc0003aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc07fff640 1 --2- 172.21.15.46:0/3076925267 >> v2:172.21.15.46:3300/0 conn(0x7efc0003a720 0x7efc0003aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc07fff640 1 -- 172.21.15.46:0/3076925267 >> v1:172.21.15.161:6789/0 conn(0x7efc0000c330 legacy=0x7efc000ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7efc07fff640 1 -- 172.21.15.46:0/3076925267 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc0000c710 con 0x7efc000214a0 2023-12-16T21:42:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7f2f32ffd640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071b40 0x7f2f34071120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c450 tx=0x7f2f2800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fccb759e640 1 -- 172.21.15.46:0/3291957619 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb80b0540 msgr2=0x7fccb8071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fccb759e640 1 --2- 172.21.15.46:0/3291957619 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb80b0540 0x7fccb8071b40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fccb0009d90 tx=0x7fccb001c910 comp rx=0 tx=0).stop 2023-12-16T21:42:46.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf82d9d640 1 -- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6c005190 msgr2=0x7fcf6c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf6c005190 0x7fcf6c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf82d9d640 1 -- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf6c0047d0 msgr2=0x7fcf6c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf6c0047d0 0x7fcf6c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7fcf82d9d640 1 -- 172.21.15.46:0/1040353865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8406cce0 con 0x7fcf6c003980 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e359e640 1 -- 172.21.15.46:0/3524783978 shutdown_connections 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e359e640 1 --2- 172.21.15.46:0/3524783978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc039fc0 0x7ff8dc0c9120 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e359e640 1 --2- 172.21.15.46:0/3524783978 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc0c2f00 0x7ff8dc0c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff8e359e640 1 -- 172.21.15.46:0/3524783978 >> 172.21.15.46:0/3524783978 conn(0x7ff8dc03aba0 msgr2=0x7ff8dc005e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f910bfff640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9104036d60 0x7f91040c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- 172.21.15.46:0/3291957619 shutdown_connections 2023-12-16T21:42:46.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 --2- 172.21.15.46:0/3291957619 >> v2:172.21.15.161:3300/0 conn(0x7fccb8072080 0x7fccb8146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 --2- 172.21.15.46:0/3291957619 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb80b0540 0x7fccb8071b40 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- 172.21.15.46:0/3291957619 >> 172.21.15.46:0/3291957619 conn(0x7fccb80b7ca0 msgr2=0x7fccb80b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.012 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 -- 172.21.15.46:0/3524783978 shutdown_connections 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f910affd640 1 -- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9104036980 msgr2=0x7f91040c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f910affd640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9104036980 0x7f91040c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f910affd640 1 -- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9104036d60 msgr2=0x7f91040c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f910affd640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9104036d60 0x7f91040c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f910affd640 1 -- 172.21.15.46:0/2100774795 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9100009420 con 0x7f91040348a0 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b37fe640 1 -- 172.21.15.46:0/3565742914 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a80ab070 con 0x7ff9b40b4f30 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.983+0000 7ff9b37fe640 1 -- 172.21.15.46:0/3565742914 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80a7d20 con 0x7ff9b40b4f30 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 -- 172.21.15.46:0/3565742914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 msgr2=0x7ff9b40b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 --2- 172.21.15.46:0/3565742914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 0x7ff9b40b09f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80087f0 tx=0x7ff9a801e8f0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf6c003980 0x7fcf6c003da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009980 tx=0x7fcf7401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6312b9113b77a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- 172.21.15.46:0/3291957619 shutdown_connections 2023-12-16T21:42:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f910affd640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91040348a0 0x7f91040c97e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f910c02cef0 tx=0x7f910c067950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c06bb20 con 0x7f91040348a0 2023-12-16T21:42:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f91116da640 1 -- 172.21.15.46:0/2100774795 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91040cacb0 con 0x7f91040348a0 2023-12-16T21:42:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c06bca0 con 0x7f91040348a0 2023-12-16T21:42:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c092640 con 0x7f91040348a0 2023-12-16T21:42:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 -- 172.21.15.46:0/3565742914 shutdown_connections 2023-12-16T21:42:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 --2- 172.21.15.46:0/3565742914 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 0x7ff9b40b09f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 -- 172.21.15.46:0/3565742914 >> 172.21.15.46:0/3565742914 conn(0x7ff9b406a580 msgr2=0x7ff9b406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f0cf9b0e640 1 --2- 172.21.15.46:0/2120562414 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec03ab00 0x7f0cec03af40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0014420 tx=0x7f0cf0025460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc6d5fbf0f9fcc64 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 -- 172.21.15.46:0/3524783978 wait complete. 2023-12-16T21:42:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 Processor -- start 2023-12-16T21:42:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- 172.21.15.46:0/3291957619 wait complete. 2023-12-16T21:42:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 Processor -- start 2023-12-16T21:42:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 -- 172.21.15.46:0/3565742914 shutdown_connections 2023-12-16T21:42:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f280074e0 con 0x7f2f34071b40 2023-12-16T21:42:46.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28007640 con 0x7f2f34071b40 2023-12-16T21:42:46.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28004a30 con 0x7f2f34071b40 2023-12-16T21:42:46.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f91116da640 1 -- 172.21.15.46:0/2100774795 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91040c9d20 con 0x7f91040348a0 2023-12-16T21:42:46.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 -- start start 2023-12-16T21:42:46.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f91116da640 1 -- 172.21.15.46:0/2100774795 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9104005540 con 0x7f91040348a0 2023-12-16T21:42:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7efc07fff640 1 --2- 172.21.15.46:0/3076925267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc000214a0 0x7efc000218f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc0003a1d0 tx=0x7efbfc01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8738b4fd1b041e61 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3413ef50 con 0x7f2f34071b40 2023-12-16T21:42:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f34140000 con 0x7f2f34071b40 2023-12-16T21:42:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2f340b26a0 con 0x7f2f34071b40 2023-12-16T21:42:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f0cf930d640 1 -- 172.21.15.46:0/2120562414 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00b5640 con 0x7f0cec03ab00 2023-12-16T21:42:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f0cf930d640 1 -- 172.21.15.46:0/2120562414 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf00b57a0 con 0x7f0cec03ab00 2023-12-16T21:42:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7f0cf930d640 1 -- 172.21.15.46:0/2120562414 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00b5b30 con 0x7f0cec03ab00 2023-12-16T21:42:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7efc067fc640 1 -- 172.21.15.46:0/3076925267 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0ac620 con 0x7efc000214a0 2023-12-16T21:42:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7efc067fc640 1 -- 172.21.15.46:0/3076925267 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0ac780 con 0x7efc000214a0 2023-12-16T21:42:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7efc067fc640 1 -- 172.21.15.46:0/3076925267 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0acb10 con 0x7efc000214a0 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 -- 172.21.15.46:0/3565742914 wait complete. 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 Processor -- start 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- start start 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb8072080 0x7fccb81e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb80b0540 0x7fccb81e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccb80b87c0 0x7fccb814a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb8149fe0 con 0x7fccb80b0540 2023-12-16T21:42:46.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81e7640 con 0x7fccb80b87c0 2023-12-16T21:42:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb759e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb81e7920 con 0x7fccb8072080 2023-12-16T21:42:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8dc038e50 0x7ff8dc03d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc039fc0 0x7ff8dc1713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc0c2f00 0x7ff8dc171910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc0ca590 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc16b970 con 0x7ff8dc038e50 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff8e359e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc0c6000 con 0x7ff8dc039fc0 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7efc0cc73640 1 -- 172.21.15.46:0/3076925267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc000214a0 msgr2=0x7efc000218f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7efc0cc73640 1 --2- 172.21.15.46:0/3076925267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc000214a0 0x7efc000218f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc0003a1d0 tx=0x7efbfc01c3c0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 -- 172.21.15.46:0/3076925267 shutdown_connections 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 --2- 172.21.15.46:0/3076925267 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc000214a0 0x7efc000218f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 --2- 172.21.15.46:0/3076925267 >> v2:172.21.15.46:3300/0 conn(0x7efc0003a720 0x7efc0003aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 -- 172.21.15.46:0/3076925267 >> 172.21.15.46:0/3076925267 conn(0x7efc0000b5b0 msgr2=0x7efc0000b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 -- start start 2023-12-16T21:42:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b40aec40 0x7ff9b4136b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7ff9baedf640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40b4300 0x7ff9b41e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9baedf640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 0x7ff9b41e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9baedf640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b413fbe0 con 0x7ff9b40aec40 2023-12-16T21:42:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9baedf640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b413fec0 con 0x7ff9b40b4f30 2023-12-16T21:42:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9baedf640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b413f100 con 0x7ff9b40b4300 2023-12-16T21:42:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e1d9b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc039fc0 0x7ff8dc1713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e1d9b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc039fc0 0x7ff8dc1713d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50522/0 (socket says 172.21.15.46:50522) 2023-12-16T21:42:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e1d9b640 1 -- 172.21.15.46:0/2834116498 learned_addr learned my addr 172.21.15.46:0/2834116498 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 -- 172.21.15.46:0/2120562414 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec03ab00 msgr2=0x7f0cec03af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 --2- 172.21.15.46:0/2120562414 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec03ab00 0x7f0cec03af40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0014420 tx=0x7f0cf0025460 comp rx=0 tx=0).stop 2023-12-16T21:42:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccb80b87c0 0x7fccb814a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb6d9d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccb80b87c0 0x7fccb814a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:35468/0 (socket says 172.21.15.46:35468) 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb6d9d640 1 -- 172.21.15.46:0/1056846930 learned_addr learned my addr 172.21.15.46:0/1056846930 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.987+0000 7fccb5d9b640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb80b0540 0x7fccb81e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e2d9d640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc0c2f00 0x7ff8dc171910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb659c640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb8072080 0x7fccb81e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb5d9b640 1 -- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb8072080 msgr2=0x7fccb81e6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb5d9b640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb8072080 0x7fccb81e6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb5d9b640 1 -- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccb80b87c0 msgr2=0x7fccb814a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb5d9b640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccb80b87c0 0x7fccb814a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40b4300 0x7ff9b41e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb5d9b640 1 -- 172.21.15.46:0/1056846930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb0009a70 con 0x7fccb80b0540 2023-12-16T21:42:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb6d9d640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fccb80b87c0 0x7fccb814a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb659c640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fccb8072080 0x7fccb81e6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf740a8440 con 0x7fcf6c003980 2023-12-16T21:42:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40b4300 0x7ff9b41e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50526/0 (socket says 172.21.15.46:50526) 2023-12-16T21:42:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 -- 172.21.15.46:0/661299343 learned_addr learned my addr 172.21.15.46:0/661299343 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf740a85a0 con 0x7fcf6c003980 2023-12-16T21:42:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf81d9b640 1 -- 172.21.15.46:0/1040353865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf740a8950 con 0x7fcf6c003980 2023-12-16T21:42:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 -- 172.21.15.46:0/2120562414 shutdown_connections 2023-12-16T21:42:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 --2- 172.21.15.46:0/2120562414 >> v2:172.21.15.46:3300/0 conn(0x7f0cec0374d0 0x7f0cec037940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 --2- 172.21.15.46:0/2120562414 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec03ab00 0x7f0cec03af40 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 -- 172.21.15.46:0/2120562414 >> 172.21.15.46:0/2120562414 conn(0x7f0cec03ea40 msgr2=0x7f0cec03ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b947c640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 0x7ff9b41e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 -- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 msgr2=0x7ff9b41e6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 0x7ff9b41e6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 -- 172.21.15.46:0/661299343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b40aec40 msgr2=0x7ff9b4136b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9b40aec40 0x7ff9b4136b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 -- 172.21.15.46:0/661299343 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80084a0 con 0x7ff9b40b4300 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf89236640 1 -- 172.21.15.46:0/1040353865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf6c003980 msgr2=0x7fcf6c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf89236640 1 --2- 172.21.15.46:0/1040353865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf6c003980 0x7fcf6c003da0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcf74009980 tx=0x7fcf7401c8f0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf8359e640 1 -- 172.21.15.46:0/1040353865 reap_dead start 2023-12-16T21:42:46.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e259c640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8dc038e50 0x7ff8dc03d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e2d9d640 1 -- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc039fc0 msgr2=0x7ff8dc1713d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e2d9d640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff8dc039fc0 0x7ff8dc1713d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e2d9d640 1 -- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8dc038e50 msgr2=0x7ff8dc03d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e2d9d640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff8dc038e50 0x7ff8dc03d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e2d9d640 1 -- 172.21.15.46:0/2834116498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e404f7e0 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b947c640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff9b40b4f30 0x7ff9b41e6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:46.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e2d9d640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc0c2f00 0x7ff8dc171910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4068240 tx=0x7ff8e4095b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf89236640 1 -- 172.21.15.46:0/1040353865 shutdown_connections 2023-12-16T21:42:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fcf89236640 1 -- 172.21.15.46:0/1040353865 >> 172.21.15.46:0/1040353865 conn(0x7fcf840b56b0 msgr2=0x7fcf840b5ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f910c0927a0 con 0x7f91040348a0 2023-12-16T21:42:46.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2f28004b90 con 0x7f2f34071b40 2023-12-16T21:42:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 -- 172.21.15.46:0/2120562414 shutdown_connections 2023-12-16T21:42:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 -- 172.21.15.46:0/2120562414 wait complete. 2023-12-16T21:42:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 -- 172.21.15.46:0/3076925267 shutdown_connections 2023-12-16T21:42:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 -- 172.21.15.46:0/3076925267 wait complete. 2023-12-16T21:42:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b3fff640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40b4300 0x7ff9b41e64d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c7c0 tx=0x7ff9ac00cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac003b80 con 0x7ff9b40b4300 2023-12-16T21:42:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f90f37fe640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f90f8066470 0x7f90f8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f910c0fdc20 con 0x7f91040348a0 2023-12-16T21:42:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f90f37fe640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7f90f806e610 0x7f90f8070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 --> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f90f80710e0 con 0x7f90f806e610 2023-12-16T21:42:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f910c0fdf90 con 0x7f91040348a0 2023-12-16T21:42:46.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccb5d9b640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb80b0540 0x7fccb81e6fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccb00ac000 tx=0x7fccb00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e412c810 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e412c970 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e412cd20 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc0c5990 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc0ca790 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff8dc005370 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 Processor -- start 2023-12-16T21:42:46.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f0cfb311640 1 -- start start 2023-12-16T21:42:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7ff8e4135410 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f910bfff640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7f90f806e610 0x7f90f8070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f1b7fe640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2f20066540 0x7f2f20068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f280839d0 con 0x7f2f34071b40 2023-12-16T21:42:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f1b7fe640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f2f2006e760 0x7f2f20070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 --> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2f20071230 con 0x7f2f2006e760 2023-12-16T21:42:46.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f28083d80 con 0x7f2f34071b40 2023-12-16T21:42:46.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f327fc640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2f20066540 0x7f2f20068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7f2f337fe640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f2f2006e760 0x7f2f20070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 Processor -- start 2023-12-16T21:42:46.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7efc0cc73640 1 -- start start 2023-12-16T21:42:46.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 -- 172.21.15.46:0/1040353865 shutdown_connections 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc0cc73640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc0000c330 0x7efc00036150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc0cc73640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc000214a0 0x7efc00033dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc0cc73640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc0003a720 0x7efc000c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc0cc73640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc000336e0 con 0x7efc0003a720 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc0cc73640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc00035710 con 0x7efc0000c330 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc0cc73640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc000359f0 con 0x7efc000214a0 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc0003a720 0x7efc000c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc0003a720 0x7efc000c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45054/0 (socket says 172.21.15.46:45054) 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 -- 172.21.15.46:0/3547403512 learned_addr learned my addr 172.21.15.46:0/3547403512 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc077fe640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc0000c330 0x7efc00036150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f910bfff640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7f90f806e610 0x7f90f8070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8d2ffd640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff8c4066470 0x7ff8c4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8e4197b50 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8d2ffd640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7ff8c406e610 0x7ff8c4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 --> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8c40710e0 con 0x7ff8c406e610 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb00a5880 con 0x7fccb80b0540 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb00ad870 con 0x7fccb80b0540 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb00adc20 con 0x7fccb80b0540 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb813c840 con 0x7fccb80b0540 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb8143710 con 0x7fccb80b0540 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac003ce0 con 0x7ff9b40b4300 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.991+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00aa90 con 0x7ff9b40b4300 2023-12-16T21:42:46.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f910a7fc640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f90f8066470 0x7f90f8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fccb81e77b0 con 0x7fccb80b0540 2023-12-16T21:42:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8e1d9b640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7ff8c406e610 0x7ff8c4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8e259c640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff8c4066470 0x7ff8c4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 -- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc000214a0 msgr2=0x7efc00033dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-16T21:42:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7efc000214a0 0x7efc00033dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 -- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc0000c330 msgr2=0x7efc00036150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7efc0000c330 0x7efc00036150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 -- 172.21.15.46:0/3547403512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc009420 con 0x7efc0003a720 2023-12-16T21:42:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7efc07fff640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc0003a720 0x7efc000c7800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0af040 tx=0x7efbfc019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== osd.2 v2:172.21.15.46:6808/3526272141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f90f80710e0 con 0x7f90f806e610 2023-12-16T21:42:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b413f870 con 0x7ff9b40b4300 2023-12-16T21:42:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4137770 con 0x7ff9b40b4300 2023-12-16T21:42:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff9b4140030 con 0x7ff9b40b4300 2023-12-16T21:42:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8e1d9b640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7ff8c406e610 0x7ff8c4070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 -- 172.21.15.46:0/1040353865 wait complete. 2023-12-16T21:42:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfb311640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec0374d0 0x7f0cec167880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfb311640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec038810 0x7f0cec0cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfb311640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cec03ab00 0x7f0cec0c02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 Processor -- start 2023-12-16T21:42:46.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 -- start start 2023-12-16T21:42:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfb311640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0ca060 con 0x7f0cec038810 2023-12-16T21:42:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfb311640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0c9110 con 0x7f0cec03ab00 2023-12-16T21:42:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec0374d0 0x7f0cec167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfab10640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cec03ab00 0x7f0cec0c02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec0374d0 0x7f0cec167880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50530/0 (socket says 172.21.15.46:50530) 2023-12-16T21:42:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 -- 172.21.15.46:0/1141420871 learned_addr learned my addr 172.21.15.46:0/1141420871 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cec0c7730 con 0x7f0cec0374d0 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 -- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cec03ab00 msgr2=0x7f0cec0c02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cec03ab00 0x7f0cec0c02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 -- 172.21.15.46:0/1141420871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec038810 msgr2=0x7f0cec0cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cec038810 0x7f0cec0cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 -- 172.21.15.46:0/1141420871 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf000fd10 con 0x7f0cec0374d0 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8414eaf0 0x7fcf841f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf84142830 0x7fcf841eca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.046 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf841f07c0 0x7fcf841ecf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841443f0 con 0x7fcf84142830 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841efb10 con 0x7fcf8414eaf0 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf89236640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf841ed4b0 con 0x7fcf841f07c0 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf84142830 0x7fcf841eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf8259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf84142830 0x7fcf841eca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45084/0 (socket says 172.21.15.46:45084) 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf8259c640 1 -- 172.21.15.46:0/557165652 learned_addr learned my addr 172.21.15.46:0/557165652 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fccb00b4070 con 0x7fccb80b0540 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccaeffd640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcc98066540 0x7fcc98068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccb011f000 con 0x7fccb80b0540 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccaeffd640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7fcc9806e6e0 0x7fcc98070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 --> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc980711b0 con 0x7fcc9806e6e0 2023-12-16T21:42:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7f0cfa30f640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec0374d0 0x7f0cec167880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4079da0 tx=0x7f0cf4067890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8414eaf0 0x7fcf841f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7fcf8359e640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf841f07c0 0x7fcf841ecf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff8e259c640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff8c4066470 0x7ff8c4068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40011f0 tx=0x7ff8d4007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.995+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7ff9ac00abf0 con 0x7ff9b40b4300 2023-12-16T21:42:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0b5610 con 0x7efc0003a720 2023-12-16T21:42:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbfc0b5770 con 0x7efc0003a720 2023-12-16T21:42:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbfc0b5b20 con 0x7efc0003a720 2023-12-16T21:42:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fccb659c640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcc98066540 0x7fcc98068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.049 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fccb6d9d640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7fcc9806e6e0 0x7fcc98070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fcf8259c640 1 -- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf841f07c0 msgr2=0x7fcf841ecf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fcf8259c640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcf841f07c0 0x7fcf841ecf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fcf8259c640 1 -- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8414eaf0 msgr2=0x7fcf841f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fcf8259c640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcf8414eaf0 0x7fcf841f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fcf8259c640 1 -- 172.21.15.46:0/557165652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8414eed0 con 0x7fcf84142830 2023-12-16T21:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fcf8259c640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf84142830 0x7fcf841eca30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009f90 tx=0x7fcf7c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.050 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf406aa50 con 0x7f0cec0374d0 2023-12-16T21:42:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf406ac30 con 0x7f0cec0374d0 2023-12-16T21:42:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf4072400 con 0x7f0cec0374d0 2023-12-16T21:42:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f2f337fe640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f2f2006e760 0x7f2f20070b80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f2f327fc640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2f20066540 0x7f2f20068a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f24000f70 tx=0x7f2f240079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fccb659c640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcc98066540 0x7fcc98068a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcca0000f70 tx=0x7fcca0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fccb6d9d640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7fcc9806e6e0 0x7fcc98070b00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc000338f0 con 0x7efc0003a720 2023-12-16T21:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc00039860 con 0x7efc0003a720 2023-12-16T21:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efbe8000fd0 con 0x7efc0003a720 2023-12-16T21:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cec0c7a10 con 0x7f0cec0374d0 2023-12-16T21:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cec0c7f30 con 0x7f0cec0374d0 2023-12-16T21:42:46.052 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0cec0cc0e0 con 0x7f0cec0374d0 2023-12-16T21:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccb00ec6c0 con 0x7fccb80b0540 2023-12-16T21:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== osd.7 v2:172.21.15.161:6816/2824550480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcc980711b0 con 0x7fcc9806e6e0 2023-12-16T21:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7ff9b17fa640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff9a00664a0 0x7ff9a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac0846e0 con 0x7ff9b40b4300 2023-12-16T21:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7ff9b17fa640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7ff9a006e6c0 0x7ff9a0070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 --> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff9a0071190 con 0x7ff9a006e6c0 2023-12-16T21:42:46.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9ac084a90 con 0x7ff9b40b4300 2023-12-16T21:42:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7ff9b947c640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7ff9a006e6c0 0x7ff9a0070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7ff9b947c640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7ff9a006e6c0 0x7ff9a0070ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7ff9b8c7b640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff9a00664a0 0x7ff9a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== osd.0 v2:172.21.15.46:6800/1629025463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff9a0071190 con 0x7ff9a006e6c0 2023-12-16T21:42:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== osd.1 v2:172.21.15.46:6824/491328631 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff8c40710e0 con 0x7ff8c406e610 2023-12-16T21:42:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e4165190 con 0x7ff8dc0c2f00 2023-12-16T21:42:46.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7f910a7fc640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f90f8066470 0x7f90f8068930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91000af000 tx=0x7f91000b6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7efbfc0b3040 con 0x7efc0003a720 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7efbec066540 0x7efbec068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efbfc11e590 con 0x7efc0003a720 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7efbec06e760 0x7efbec070b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 --> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efbec071230 con 0x7efbec06e760 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7efbfc11e940 con 0x7efc0003a720 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efc06ffd640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7efbec06e760 0x7efbec070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:45.999+0000 7efc077fe640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7efbec066540 0x7efbec068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7efc06ffd640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7efbec06e760 0x7efbec070b80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7efc077fe640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7efbec066540 0x7efbec068a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005890 tx=0x7efbf4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.003+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== osd.3 v2:172.21.15.46:6809/2268558465 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7efbec071230 con 0x7efbec06e760 2023-12-16T21:42:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== osd.5 v2:172.21.15.161:6808/1602903861 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2f20071230 con 0x7f2f2006e760 2023-12-16T21:42:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf6b7fe640 1 -- 172.21.15.46:0/557165652 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c00e040 con 0x7fcf84142830 2023-12-16T21:42:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf6b7fe640 1 -- 172.21.15.46:0/557165652 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7c00f040 con 0x7fcf84142830 2023-12-16T21:42:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf6b7fe640 1 -- 172.21.15.46:0/557165652 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7c003dd0 con 0x7fcf84142830 2023-12-16T21:42:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf84142d40 con 0x7fcf84142830 2023-12-16T21:42:46.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf841ec520 con 0x7fcf84142830 2023-12-16T21:42:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f0cf4072560 con 0x7f0cec0374d0 2023-12-16T21:42:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0ceaffd640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd80664c0 0x7f0cd8068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cf40fdc50 con 0x7f0cec0374d0 2023-12-16T21:42:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0ceaffd640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f0cd806e6e0 0x7f0cd8070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 --> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0cd80711b0 con 0x7f0cd806e6e0 2023-12-16T21:42:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf40fe000 con 0x7f0cec0374d0 2023-12-16T21:42:46.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0cf9b0e640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd80664c0 0x7f0cd8068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0cfab10640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f0cd806e6e0 0x7f0cd8070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf6b7fe640 1 -- 172.21.15.46:0/557165652 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fcf7c004030 con 0x7fcf84142830 2023-12-16T21:42:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf6b7fe640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcf70066570 0x7fcf70068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf6b7fe640 1 -- 172.21.15.46:0/557165652 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf7c084960 con 0x7fcf84142830 2023-12-16T21:42:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcf70066570 0x7fcf70068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7ff9b8c7b640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff9a00664a0 0x7ff9a0068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80b2000 tx=0x7ff9a80a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0cf9b0e640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd80664c0 0x7f0cd8068980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00b7000 tx=0x7f0cf0004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7f0cfab10640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f0cd806e6e0 0x7f0cd8070b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.007+0000 7fcf82d9d640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcf70066570 0x7fcf70068a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf740af000 tx=0x7fcf740b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.011+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== osd.4 v2:172.21.15.161:6800/3753525888 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f0cd80711b0 con 0x7f0cd806e6e0 2023-12-16T21:42:46.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.015+0000 7fcf89236640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7fcf841516b0 0x7fcf841435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.015+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 --> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf840679e0 con 0x7fcf841516b0 2023-12-16T21:42:46.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.015+0000 7fcf8359e640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7fcf841516b0 0x7fcf841435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.015+0000 7fcf8359e640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7fcf841516b0 0x7fcf841435d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.015+0000 7fcf6b7fe640 1 -- 172.21.15.46:0/557165652 <== osd.6 v2:172.21.15.161:6824/2291774553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcf840679e0 con 0x7fcf841516b0 2023-12-16T21:42:46.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 --> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff8dc0c6140 con 0x7ff8c406e610 2023-12-16T21:42:46.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7ff8d2ffd640 1 -- 172.21.15.46:0/2834116498 <== osd.1 v2:172.21.15.46:6824/491328631 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff8dc0c6140 con 0x7ff8c406e610 2023-12-16T21:42:46.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 --> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2f341425b0 con 0x7f2f2006e760 2023-12-16T21:42:46.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7f2f1b7fe640 1 -- 172.21.15.46:0/2687436700 <== osd.5 v2:172.21.15.161:6808/1602903861 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2f2c009450 con 0x7f2f2006e760 2023-12-16T21:42:46.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 --> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fccb80b7ab0 con 0x7fcc9806e6e0 2023-12-16T21:42:46.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7f91116da640 1 -- 172.21.15.46:0/2100774795 --> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f91041698b0 con 0x7f90f806e610 2023-12-16T21:42:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7ff8c406e610 msgr2=0x7ff8c4070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.023+0000 7ff8e359e640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7ff8c406e610 0x7ff8c4070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff8c4066470 msgr2=0x7ff8c4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff8c4066470 0x7ff8c4068930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40011f0 tx=0x7ff8d4007d40 comp rx=0 tx=0).stop 2023-12-16T21:42:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc0c2f00 msgr2=0x7ff8dc171910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 --2- 172.21.15.46:0/2834116498 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8dc0c2f00 0x7ff8dc171910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4068240 tx=0x7ff8e4095b10 comp rx=0 tx=0).stop 2023-12-16T21:42:46.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f37fe640 1 -- 172.21.15.46:0/2100774795 <== osd.2 v2:172.21.15.46:6808/3526272141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f91041698b0 con 0x7f90f806e610 2023-12-16T21:42:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e2d9d640 1 -- 172.21.15.46:0/2834116498 reap_dead start 2023-12-16T21:42:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 shutdown_connections 2023-12-16T21:42:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 >> 172.21.15.46:0/2834116498 conn(0x7ff8dc03aba0 msgr2=0x7ff8dc0beb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 shutdown_connections 2023-12-16T21:42:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7ff8e359e640 1 -- 172.21.15.46:0/2834116498 wait complete. 2023-12-16T21:42:46.064 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 --> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efbe8002cc0 con 0x7efbec06e760 2023-12-16T21:42:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efbf3fff640 1 -- 172.21.15.46:0/3547403512 <== osd.3 v2:172.21.15.46:6809/2268558465 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efbe8002cc0 con 0x7efbec06e760 2023-12-16T21:42:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 -- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7f90f806e610 msgr2=0x7f90f8070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7f90f806e610 0x7f90f8070a30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 -- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f90f8066470 msgr2=0x7f90f8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f90f8066470 0x7f90f8068930 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f91000af000 tx=0x7f91000b6040 comp rx=0 tx=0).stop 2023-12-16T21:42:46.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 -- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91040348a0 msgr2=0x7f91040c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 --2- 172.21.15.46:0/2100774795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f91040348a0 0x7f91040c97e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f910c02cef0 tx=0x7f910c067950 comp rx=0 tx=0).stop 2023-12-16T21:42:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f910bfff640 1 -- 172.21.15.46:0/2100774795 reap_dead start 2023-12-16T21:42:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 -- 172.21.15.46:0/2100774795 shutdown_connections 2023-12-16T21:42:46.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7f90f17fa640 1 -- 172.21.15.46:0/2100774795 >> 172.21.15.46:0/2100774795 conn(0x7f910403a800 msgr2=0x7f910403abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7efbec06e760 msgr2=0x7efbec070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7efbec06e760 0x7efbec070b80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7efbec066540 msgr2=0x7efbec068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7efbec066540 0x7efbec068a00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7efbf4005890 tx=0x7efbf4009040 comp rx=0 tx=0).stop 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc0003a720 msgr2=0x7efc000c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 --2- 172.21.15.46:0/3547403512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efc0003a720 0x7efc000c7800 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7efbfc0af040 tx=0x7efbfc019160 comp rx=0 tx=0).stop 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc07fff640 1 -- 172.21.15.46:0/3547403512 reap_dead start 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.027+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 shutdown_connections 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 >> 172.21.15.46:0/3547403512 conn(0x7efc0000b5b0 msgr2=0x7efc000068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccaeffd640 1 -- 172.21.15.46:0/1056846930 <== osd.7 v2:172.21.15.161:6816/2824550480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fccb80b7ab0 con 0x7fcc9806e6e0 2023-12-16T21:42:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 shutdown_connections 2023-12-16T21:42:46.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7efc0cc73640 1 -- 172.21.15.46:0/3547403512 wait complete. 2023-12-16T21:42:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7fcc9806e6e0 msgr2=0x7fcc98070b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7fcc9806e6e0 0x7fcc98070b00 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f90f17fa640 1 -- 172.21.15.46:0/2100774795 shutdown_connections 2023-12-16T21:42:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcc98066540 msgr2=0x7fcc98068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcc98066540 0x7fcc98068a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcca0000f70 tx=0x7fcca0009040 comp rx=0 tx=0).stop 2023-12-16T21:42:46.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb80b0540 msgr2=0x7fccb81e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 --2- 172.21.15.46:0/1056846930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fccb80b0540 0x7fccb81e6fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fccb00ac000 tx=0x7fccb00196b0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb6d9d640 1 -- 172.21.15.46:0/1056846930 reap_dead start 2023-12-16T21:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 shutdown_connections 2023-12-16T21:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 >> 172.21.15.46:0/1056846930 conn(0x7fccb80b7ca0 msgr2=0x7fccb80b4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f90f17fa640 1 -- 172.21.15.46:0/2100774795 wait complete. 2023-12-16T21:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 shutdown_connections 2023-12-16T21:42:46.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7fccb759e640 1 -- 172.21.15.46:0/1056846930 wait complete. 2023-12-16T21:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 --> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff9b41401a0 con 0x7ff9a006e6c0 2023-12-16T21:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7ff9b17fa640 1 -- 172.21.15.46:0/661299343 <== osd.0 v2:172.21.15.46:6800/1629025463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff9b41401a0 con 0x7ff9a006e6c0 2023-12-16T21:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f2f2006e760 msgr2=0x7f2f20070b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f2f2006e760 0x7f2f20070b80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2f20066540 msgr2=0x7f2f20068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2f20066540 0x7f2f20068a00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f24000f70 tx=0x7f2f240079c0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071b40 msgr2=0x7f2f34071120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 --2- 172.21.15.46:0/2687436700 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2f34071b40 0x7f2f34071120 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c450 tx=0x7f2f2800c910 comp rx=0 tx=0).stop 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f337fe640 1 -- 172.21.15.46:0/2687436700 reap_dead start 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 shutdown_connections 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.031+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 >> 172.21.15.46:0/2687436700 conn(0x7f2f340b7ca0 msgr2=0x7f2f340b65e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.035+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 shutdown_connections 2023-12-16T21:42:46.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.035+0000 7f2f33fff640 1 -- 172.21.15.46:0/2687436700 wait complete. 2023-12-16T21:42:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7ff9a006e6c0 msgr2=0x7ff9a0070ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7ff9a006e6c0 0x7ff9a0070ae0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff9a00664a0 msgr2=0x7ff9a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff9a00664a0 0x7ff9a0068960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80b2000 tx=0x7ff9a80a9070 comp rx=0 tx=0).stop 2023-12-16T21:42:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40b4300 msgr2=0x7ff9b41e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 --2- 172.21.15.46:0/661299343 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff9b40b4300 0x7ff9b41e64d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00c7c0 tx=0x7ff9ac00cc80 comp rx=0 tx=0).stop 2023-12-16T21:42:46.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9b947c640 1 -- 172.21.15.46:0/661299343 reap_dead start 2023-12-16T21:42:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 shutdown_connections 2023-12-16T21:42:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 >> 172.21.15.46:0/661299343 conn(0x7ff9b406a580 msgr2=0x7ff9b4071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 shutdown_connections 2023-12-16T21:42:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7ff9baedf640 1 -- 172.21.15.46:0/661299343 wait complete. 2023-12-16T21:42:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 --> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0cec0cc250 con 0x7f0cd806e6e0 2023-12-16T21:42:46.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 --> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf841effa0 con 0x7fcf841516b0 2023-12-16T21:42:46.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7f0ceaffd640 1 -- 172.21.15.46:0/1141420871 <== osd.4 v2:172.21.15.161:6800/3753525888 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0cec0cc250 con 0x7f0cd806e6e0 2023-12-16T21:42:46.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.039+0000 7fcf6b7fe640 1 -- 172.21.15.46:0/557165652 <== osd.6 v2:172.21.15.161:6824/2291774553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf841effa0 con 0x7fcf841516b0 2023-12-16T21:42:46.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f0cd806e6e0 msgr2=0x7f0cd8070b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f0cd806e6e0 0x7f0cd8070b00 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd80664c0 msgr2=0x7f0cd8068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd80664c0 0x7f0cd8068980 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00b7000 tx=0x7f0cf0004740 comp rx=0 tx=0).stop 2023-12-16T21:42:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec0374d0 msgr2=0x7f0cec167880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 --2- 172.21.15.46:0/1141420871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cec0374d0 0x7f0cec167880 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4079da0 tx=0x7f0cf4067890 comp rx=0 tx=0).stop 2023-12-16T21:42:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfab10640 1 -- 172.21.15.46:0/1141420871 reap_dead start 2023-12-16T21:42:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 shutdown_connections 2023-12-16T21:42:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 >> 172.21.15.46:0/1141420871 conn(0x7f0cec03ea40 msgr2=0x7f0cec040990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 shutdown_connections 2023-12-16T21:42:46.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.043+0000 7f0cfb311640 1 -- 172.21.15.46:0/1141420871 wait complete. 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7fcf841516b0 msgr2=0x7fcf841435d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7fcf841516b0 0x7fcf841435d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcf70066570 msgr2=0x7fcf70068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcf70066570 0x7fcf70068a30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcf740af000 tx=0x7fcf740b3040 comp rx=0 tx=0).stop 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf84142830 msgr2=0x7fcf841eca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 --2- 172.21.15.46:0/557165652 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcf84142830 0x7fcf841eca30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009f90 tx=0x7fcf7c00f580 comp rx=0 tx=0).stop 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf8359e640 1 -- 172.21.15.46:0/557165652 reap_dead start 2023-12-16T21:42:46.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 shutdown_connections 2023-12-16T21:42:46.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 >> 172.21.15.46:0/557165652 conn(0x7fcf840b56b0 msgr2=0x7fcf841500c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 shutdown_connections 2023-12-16T21:42:46.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.051+0000 7fcf89236640 1 -- 172.21.15.46:0/557165652 wait complete. 2023-12-16T21:42:46.079 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-16T21:42:46.080 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-16T21:42:46.081 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-16T21:42:46.081 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-16T21:42:46.082 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-16T21:42:46.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-16T21:42:46.083 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-16T21:42:46.083 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-16T21:42:46.084 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-16T21:42:46.084 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-16T21:42:46.085 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-16T21:42:46.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-16T21:42:46.086 INFO:teuthology.orchestra.run.smithi046.stdout:55834574853 2023-12-16T21:42:46.087 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-16T21:42:46.088 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-12-16T21:42:46.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-16T21:42:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 Processor -- start 2023-12-16T21:42:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 -- start start 2023-12-16T21:42:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f92400b0310 0x7f92400b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92400b4440 0x7f92400b2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92400b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92400af240 con 0x7f92400b4440 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9240072440 con 0x7f92400b0310 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f924872f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92400b1dd0 con 0x7f92400b2c80 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9245cca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92400b4440 0x7f92400b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9245cca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f92400b4440 0x7f92400b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45086/0 (socket says 172.21.15.46:45086) 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9245cca640 1 -- 172.21.15.46:0/1830610849 learned_addr learned my addr 172.21.15.46:0/1830610849 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9246ccc640 1 --2- 172.21.15.46:0/1830610849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92400b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f92464cb640 1 --2- 172.21.15.46:0/1830610849 >> v2:172.21.15.161:3300/0 conn(0x7f92400b0310 0x7f92400b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9246ccc640 1 -- 172.21.15.46:0/1830610849 >> v2:172.21.15.161:3300/0 conn(0x7f92400b0310 msgr2=0x7f92400b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9246ccc640 1 --2- 172.21.15.46:0/1830610849 >> v2:172.21.15.161:3300/0 conn(0x7f92400b0310 0x7f92400b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9246ccc640 1 -- 172.21.15.46:0/1830610849 >> v2:172.21.15.46:3300/0 conn(0x7f92400b4440 msgr2=0x7f92400b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9246ccc640 1 --2- 172.21.15.46:0/1830610849 >> v2:172.21.15.46:3300/0 conn(0x7f92400b4440 0x7f92400b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.199+0000 7f9246ccc640 1 -- 172.21.15.46:0/1830610849 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92400af660 con 0x7f92400b2c80 2023-12-16T21:42:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f92464cb640 1 --2- 172.21.15.46:0/1830610849 >> v2:172.21.15.161:3300/0 conn(0x7f92400b0310 0x7f92400b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f9246ccc640 1 --2- 172.21.15.46:0/1830610849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92400b30f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f92400b5900 tx=0x7f923801c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f99b32d091ed831 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f92454c9640 1 -- 172.21.15.46:0/1830610849 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800d2d0 con 0x7f92400b2c80 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f92454c9640 1 -- 172.21.15.46:0/1830610849 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800d430 con 0x7f92400b2c80 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f92454c9640 1 -- 172.21.15.46:0/1830610849 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92380a5980 con 0x7f92400b2c80 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f924872f640 1 -- 172.21.15.46:0/1830610849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 msgr2=0x7f92400b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f924872f640 1 --2- 172.21.15.46:0/1830610849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92400b30f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f92400b5900 tx=0x7f923801c8c0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f924872f640 1 -- 172.21.15.46:0/1830610849 shutdown_connections 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f924872f640 1 --2- 172.21.15.46:0/1830610849 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92400b30f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f924872f640 1 --2- 172.21.15.46:0/1830610849 >> v2:172.21.15.46:3300/0 conn(0x7f92400b4440 0x7f92400b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f924872f640 1 --2- 172.21.15.46:0/1830610849 >> v2:172.21.15.161:3300/0 conn(0x7f92400b0310 0x7f92400b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.203+0000 7f924872f640 1 -- 172.21.15.46:0/1830610849 >> 172.21.15.46:0/1830610849 conn(0x7f924006a580 msgr2=0x7f924006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 -- 172.21.15.46:0/1830610849 shutdown_connections 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 -- 172.21.15.46:0/1830610849 wait complete. 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 Processor -- start 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 -- start start 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92400b0310 0x7f9240158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92401ee7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92400b4440 0x7f92401eece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9240158c30 con 0x7f92400b4440 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92400ad290 con 0x7f92400b0310 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f924872f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9240158920 con 0x7f92400b2c80 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f9246ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92400b4440 0x7f92401eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f9246ccc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92400b4440 0x7f92401eece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45092/0 (socket says 172.21.15.46:45092) 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f9246ccc640 1 -- 172.21.15.46:0/3900506985 learned_addr learned my addr 172.21.15.46:0/3900506985 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f92464cb640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92400b0310 0x7f9240158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.207+0000 7f9245cca640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92401ee7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9246ccc640 1 -- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 msgr2=0x7f92401ee7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9246ccc640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92401ee7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9246ccc640 1 -- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92400b0310 msgr2=0x7f9240158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9246ccc640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92400b0310 0x7f9240158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9246ccc640 1 -- 172.21.15.46:0/3900506985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923800f050 con 0x7f92400b4440 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9246ccc640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92400b4440 0x7f92401eece0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f923801af70 tx=0x7f9238019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.219 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9236ffd640 1 -- 172.21.15.46:0/3900506985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800b940 con 0x7f92400b4440 2023-12-16T21:42:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f924872f640 1 -- 172.21.15.46:0/3900506985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92401575c0 con 0x7f92400b4440 2023-12-16T21:42:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f924872f640 1 -- 172.21.15.46:0/3900506985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240157290 con 0x7f92400b4440 2023-12-16T21:42:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9236ffd640 1 -- 172.21.15.46:0/3900506985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800baa0 con 0x7f92400b4440 2023-12-16T21:42:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9236ffd640 1 -- 172.21.15.46:0/3900506985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800be50 con 0x7f92400b4440 2023-12-16T21:42:46.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.211+0000 7f9236ffd640 1 -- 172.21.15.46:0/3900506985 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f92380c9480 con 0x7f92400b4440 2023-12-16T21:42:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.215+0000 7f9236ffd640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9224066540 0x7f9224068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.215+0000 7f9236ffd640 1 -- 172.21.15.46:0/3900506985 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f923812f880 con 0x7f92400b4440 2023-12-16T21:42:46.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.215+0000 7f92464cb640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9224066540 0x7f9224068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.222 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.215+0000 7f92464cb640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9224066540 0x7f9224068a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9230011fd0 tx=0x7f9230008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.223 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.215+0000 7f924872f640 1 -- 172.21.15.46:0/3900506985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228005260 con 0x7f92400b4440 2023-12-16T21:42:46.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.223+0000 7f9236ffd640 1 -- 172.21.15.46:0/3900506985 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f92380fcfd0 con 0x7f92400b4440 2023-12-16T21:42:46.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d6592640 1 Processor -- start 2023-12-16T21:42:46.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d6592640 1 -- start start 2023-12-16T21:42:46.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d6592640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4300 0x7f20d00b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d6592640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20d00aec40 0x7f20d0071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d6592640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20d00af700 con 0x7f20d00b4f30 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d6592640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20d00b1e50 con 0x7f20d00aec40 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d6592640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00b5310 con 0x7f20d00b4300 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20cffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f20d00b4f30 0x7f20d0142400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43060/0 (socket says 172.21.15.46:43060) 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20cffff640 1 -- 172.21.15.46:0/331952722 learned_addr learned my addr 172.21.15.46:0/331952722 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20cf7fe640 1 --2- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4300 0x7f20d00b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20d4b2f640 1 --2- 172.21.15.46:0/331952722 >> v2:172.21.15.161:3300/0 conn(0x7f20d00aec40 0x7f20d0071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.275+0000 7f20ceffd640 1 -- 172.21.15.46:0/331952722 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371354311 0 0) 0x7f20d00af700 con 0x7f20d00b4f30 2023-12-16T21:42:46.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.279+0000 7f20d4b2f640 1 -- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4300 msgr2=0x7f20d00b09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.279+0000 7f20d4b2f640 1 --2- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4300 0x7f20d00b09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.279+0000 7f20d4b2f640 1 -- 172.21.15.46:0/331952722 >> v1:172.21.15.46:6789/0 conn(0x7f20d00b4f30 legacy=0x7f20d0142400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.279+0000 7f20d4b2f640 1 -- 172.21.15.46:0/331952722 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d00af960 con 0x7f20d00aec40 2023-12-16T21:42:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.279+0000 7f20cf7fe640 1 --2- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4300 0x7f20d00b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.279+0000 7f20d4b2f640 1 --2- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 0x7f20d0071ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20d00b1ff0 tx=0x7f20c401d5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854cf9acec060749 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.288 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20ceffd640 1 -- 172.21.15.46:0/331952722 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40a76f0 con 0x7f20d00aec40 2023-12-16T21:42:46.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20ceffd640 1 -- 172.21.15.46:0/331952722 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c40a7850 con 0x7f20d00aec40 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20ceffd640 1 -- 172.21.15.46:0/331952722 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c40a7be0 con 0x7f20d00aec40 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 -- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 msgr2=0x7f20d0071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 --2- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 0x7f20d0071ab0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f20d00b1ff0 tx=0x7f20c401d5f0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 -- 172.21.15.46:0/331952722 shutdown_connections 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 --2- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 0x7f20d0071ab0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 --2- 172.21.15.46:0/331952722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4300 0x7f20d00b09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 -- 172.21.15.46:0/331952722 >> 172.21.15.46:0/331952722 conn(0x7f20d006a580 msgr2=0x7f20d006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 -- 172.21.15.46:0/331952722 shutdown_connections 2023-12-16T21:42:46.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 -- 172.21.15.46:0/331952722 wait complete. 2023-12-16T21:42:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 Processor -- start 2023-12-16T21:42:46.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.283+0000 7f20d6592640 1 -- start start 2023-12-16T21:42:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20d6592640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 0x7f20d0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20d6592640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20d00b4300 0x7f20d01daef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20d6592640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4f30 0x7f20d01db430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20d6592640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d00ae750 con 0x7f20d00b4300 2023-12-16T21:42:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20d6592640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0139e00 con 0x7f20d00aec40 2023-12-16T21:42:46.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20d6592640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0070f70 con 0x7f20d00b4f30 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20d00b4300 0x7f20d01daef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20d00b4300 0x7f20d01daef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45098/0 (socket says 172.21.15.46:45098) 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 -- 172.21.15.46:0/4089399584 learned_addr learned my addr 172.21.15.46:0/4089399584 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20d4b2f640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4f30 0x7f20d01db430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 -- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4f30 msgr2=0x7f20d01db430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4f30 0x7f20d01db430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 -- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 msgr2=0x7f20d0070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 0x7f20d0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 -- 172.21.15.46:0/4089399584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c800c300 con 0x7f20d00b4300 2023-12-16T21:42:46.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.287+0000 7f20cf7fe640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20d00b4300 0x7f20d01daef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ccb0 tx=0x7f20c8007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20ccff9640 1 -- 172.21.15.46:0/4089399584 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c80097e0 con 0x7f20d00b4300 2023-12-16T21:42:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c4009420 con 0x7f20d00b4300 2023-12-16T21:42:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d00712e0 con 0x7f20d00b4300 2023-12-16T21:42:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20ccff9640 1 -- 172.21.15.46:0/4089399584 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20c8003ba0 con 0x7f20d00b4300 2023-12-16T21:42:46.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20ccff9640 1 -- 172.21.15.46:0/4089399584 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8004030 con 0x7f20d00b4300 2023-12-16T21:42:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651c95640 1 Processor -- start 2023-12-16T21:42:46.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20ccff9640 1 -- 172.21.15.46:0/4089399584 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f20c8004210 con 0x7f20d00b4300 2023-12-16T21:42:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20bc005260 con 0x7f20d00b4300 2023-12-16T21:42:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651c95640 1 -- start start 2023-12-16T21:42:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651c95640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd644006e80 0x7fd644036510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651c95640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd6440c85d0 0x7fd6440cce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651c95640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd644036dd0 con 0x7fd6440338f0 2023-12-16T21:42:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651c95640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd644036a50 con 0x7fd644006e80 2023-12-16T21:42:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651c95640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd644029f80 con 0x7fd6440c85d0 2023-12-16T21:42:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651494640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd6440c85d0 0x7fd6440cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651494640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd6440c85d0 0x7fd6440cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:35572/0 (socket says 172.21.15.46:35572) 2023-12-16T21:42:46.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7fd651494640 1 -- 172.21.15.46:0/511822852 learned_addr learned my addr 172.21.15.46:0/511822852 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.291+0000 7f20ccff9640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f20b80666c0 0x7f20b8068b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7f20ccff9640 1 -- 172.21.15.46:0/4089399584 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20c8084c30 con 0x7f20d00b4300 2023-12-16T21:42:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7f20cffff640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f20b80666c0 0x7f20b8068b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7f20cffff640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f20b80666c0 0x7f20b8068b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f20c0000c10 tx=0x7f20c00079d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd651494640 1 -- 172.21.15.46:0/511822852 >> v2:172.21.15.46:3300/0 conn(0x7fd644006e80 msgr2=0x7fd644036510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd651494640 1 --2- 172.21.15.46:0/511822852 >> v2:172.21.15.46:3300/0 conn(0x7fd644006e80 0x7fd644036510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd651494640 1 -- 172.21.15.46:0/511822852 >> v1:172.21.15.46:6789/0 conn(0x7fd6440338f0 legacy=0x7fd644035530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd651494640 1 -- 172.21.15.46:0/511822852 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd644035c40 con 0x7fd6440c85d0 2023-12-16T21:42:46.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd651494640 1 --2- 172.21.15.46:0/511822852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440c85d0 0x7fd6440cce40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0501f0 tx=0x7fd64c096890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37c5735a291b642b server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd64b7fe640 1 -- 172.21.15.46:0/511822852 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c128070 con 0x7fd6440c85d0 2023-12-16T21:42:46.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd64b7fe640 1 -- 172.21.15.46:0/511822852 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c1244e0 con 0x7fd6440c85d0 2023-12-16T21:42:46.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.295+0000 7fd64b7fe640 1 -- 172.21.15.46:0/511822852 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c124950 con 0x7fd6440c85d0 2023-12-16T21:42:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93397c8640 1 Processor -- start 2023-12-16T21:42:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93397c8640 1 -- start start 2023-12-16T21:42:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- 172.21.15.46:0/511822852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440c85d0 msgr2=0x7fd6440cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 --2- 172.21.15.46:0/511822852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440c85d0 0x7fd6440cce40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd64c0501f0 tx=0x7fd64c096890 comp rx=0 tx=0).stop 2023-12-16T21:42:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- 172.21.15.46:0/511822852 shutdown_connections 2023-12-16T21:42:46.305 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 --2- 172.21.15.46:0/511822852 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440c85d0 0x7fd6440cce40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 --2- 172.21.15.46:0/511822852 >> v2:172.21.15.46:3300/0 conn(0x7fd644006e80 0x7fd644036510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- 172.21.15.46:0/511822852 >> 172.21.15.46:0/511822852 conn(0x7fd64403a830 msgr2=0x7fd64403ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f20ccff9640 1 -- 172.21.15.46:0/4089399584 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f20c8052120 con 0x7f20d00b4300 2023-12-16T21:42:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- 172.21.15.46:0/511822852 shutdown_connections 2023-12-16T21:42:46.306 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- 172.21.15.46:0/511822852 wait complete. 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93397c8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f933413c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93397c8640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340b0820 con 0x7f93340b13a0 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93397c8640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93340b0af0 con 0x7f93340af8d0 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93397c8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93340b1c40 con 0x7f93340b0390 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93337fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f93340b13a0 0x7f9334072280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43080/0 (socket says 172.21.15.46:43080) 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f93337fe640 1 -- 172.21.15.46:0/723530467 learned_addr learned my addr 172.21.15.46:0/723530467 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f9338fc7640 1 --2- 172.21.15.46:0/723530467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f933413c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.307 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 Processor -- start 2023-12-16T21:42:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 Processor -- start 2023-12-16T21:42:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f9332ffd640 1 -- 172.21.15.46:0/723530467 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552720775 0 0) 0x7f93340b0820 con 0x7f93340b13a0 2023-12-16T21:42:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f9332ffd640 1 -- 172.21.15.46:0/723530467 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f931c003530 con 0x7f93340b13a0 2023-12-16T21:42:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f9332ffd640 1 -- 172.21.15.46:0/723530467 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971645493 0 0) 0x7f931c003530 con 0x7f93340b13a0 2023-12-16T21:42:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f9338fc7640 1 -- 172.21.15.46:0/723530467 >> v1:172.21.15.161:6789/0 conn(0x7f93340af8d0 legacy=0x7f9334073450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f9338fc7640 1 -- 172.21.15.46:0/723530467 >> v1:172.21.15.46:6789/0 conn(0x7f93340b13a0 legacy=0x7f9334072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- start start 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd644006e80 0x7fd6440c0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440338f0 0x7fd6440c1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6440c85d0 0x7fd644168dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440c1550 con 0x7fd6440c85d0 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6440ba6a0 con 0x7fd6440338f0 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651c95640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64403d7e0 con 0x7fd644006e80 2023-12-16T21:42:46.309 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 -- start start 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 0x7f8798142de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8798071b40 0x7f8798071f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8798145b20 0x7f8798145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8798142950 con 0x7f8798145b20 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87980af660 con 0x7f8798071b40 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879e459640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87980b09b0 con 0x7f87980b05d0 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879dc58640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8798145b20 0x7f8798145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879dc58640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8798145b20 0x7f8798145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45126/0 (socket says 172.21.15.46:45126) 2023-12-16T21:42:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f879dc58640 1 -- 172.21.15.46:0/3358108641 learned_addr learned my addr 172.21.15.46:0/3358108641 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651494640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6440c85d0 0x7fd644168dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651494640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6440c85d0 0x7fd644168dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45132/0 (socket says 172.21.15.46:45132) 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7fd651494640 1 -- 172.21.15.46:0/3180488070 learned_addr learned my addr 172.21.15.46:0/3180488070 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd64bfff640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440338f0 0x7fd6440c1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd650c93640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd644006e80 0x7fd6440c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.299+0000 7f9338fc7640 1 -- 172.21.15.46:0/723530467 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933406b110 con 0x7f93340b0390 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f879dc58640 1 -- 172.21.15.46:0/3358108641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 msgr2=0x7f8798142de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-16T21:42:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f879dc58640 1 --2- 172.21.15.46:0/3358108641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 0x7f8798142de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f879dc58640 1 -- 172.21.15.46:0/3358108641 >> v2:172.21.15.161:3300/0 conn(0x7f8798071b40 msgr2=0x7f8798071f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f879dc58640 1 --2- 172.21.15.46:0/3358108641 >> v2:172.21.15.161:3300/0 conn(0x7f8798071b40 0x7f8798071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f879dc58640 1 -- 172.21.15.46:0/3358108641 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87980b19b0 con 0x7f8798145b20 2023-12-16T21:42:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd64bfff640 1 -- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd644006e80 msgr2=0x7fd6440c0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd64bfff640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd644006e80 0x7fd6440c0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd64bfff640 1 -- 172.21.15.46:0/3180488070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6440c85d0 msgr2=0x7fd644168dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd64bfff640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6440c85d0 0x7fd644168dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd64bfff640 1 -- 172.21.15.46:0/3180488070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c04ed70 con 0x7fd6440338f0 2023-12-16T21:42:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd64bfff640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440338f0 0x7fd6440c1010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c390 tx=0x7fd63800c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd6497fa640 1 -- 172.21.15.46:0/3180488070 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638003600 con 0x7fd6440338f0 2023-12-16T21:42:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd6497fa640 1 -- 172.21.15.46:0/3180488070 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd638003760 con 0x7fd6440338f0 2023-12-16T21:42:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d6a4f2640 1 Processor -- start 2023-12-16T21:42:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f879dc58640 1 --2- 172.21.15.46:0/3358108641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798145b20 0x7f8798145fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8798067920 tx=0x7f87900210c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f499f4c57cc59ea0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd6497fa640 1 -- 172.21.15.46:0/3180488070 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638003af0 con 0x7fd6440338f0 2023-12-16T21:42:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6440373e0 con 0x7fd6440338f0 2023-12-16T21:42:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6440bfdc0 con 0x7fd6440338f0 2023-12-16T21:42:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f878ffff640 1 -- 172.21.15.46:0/3358108641 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87900092d0 con 0x7f8798145b20 2023-12-16T21:42:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d6a4f2640 1 -- start start 2023-12-16T21:42:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d6a4f2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d640b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d6a4f2640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d640b5200 con 0x7f2d64146f50 2023-12-16T21:42:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d6a4f2640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d64146ac0 con 0x7f2d64140a40 2023-12-16T21:42:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d6a4f2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64146580 con 0x7f2d640b4910 2023-12-16T21:42:46.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f878ffff640 1 -- 172.21.15.46:0/3358108641 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87900ac050 con 0x7f8798145b20 2023-12-16T21:42:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f878ffff640 1 -- 172.21.15.46:0/3358108641 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8790009650 con 0x7f8798145b20 2023-12-16T21:42:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f9338fc7640 1 --2- 172.21.15.46:0/723530467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f933413c5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f932c009dc0 tx=0x7f932c01c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=835c2ec324959fa7 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d63fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d640b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d63fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d640b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50636/0 (socket says 172.21.15.46:50636) 2023-12-16T21:42:46.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d63fff640 1 -- 172.21.15.46:0/1997892935 learned_addr learned my addr 172.21.15.46:0/1997892935 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f9332ffd640 1 -- 172.21.15.46:0/723530467 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9324001680 con 0x7f93340b0390 2023-12-16T21:42:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd644005370 con 0x7fd6440338f0 2023-12-16T21:42:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 -- 172.21.15.46:0/3358108641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798145b20 msgr2=0x7f8798145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 --2- 172.21.15.46:0/3358108641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798145b20 0x7f8798145fd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8798067920 tx=0x7f87900210c0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 -- 172.21.15.46:0/3358108641 shutdown_connections 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 --2- 172.21.15.46:0/3358108641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798145b20 0x7f8798145fd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 --2- 172.21.15.46:0/3358108641 >> v2:172.21.15.161:3300/0 conn(0x7f8798071b40 0x7f8798071f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 --2- 172.21.15.46:0/3358108641 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 0x7f8798142de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 -- 172.21.15.46:0/3358108641 >> 172.21.15.46:0/3358108641 conn(0x7f87980b7ca0 msgr2=0x7f87980b80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d63fff640 1 -- 172.21.15.46:0/1997892935 >> v1:172.21.15.161:6789/0 conn(0x7f2d64140a40 legacy=0x7f2d64150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f2d63fff640 1 -- 172.21.15.46:0/1997892935 >> v1:172.21.15.46:6789/0 conn(0x7f2d64146f50 legacy=0x7f2d64140330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f2d63fff640 1 -- 172.21.15.46:0/1997892935 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64146790 con 0x7f2d640b4910 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f9332ffd640 1 -- 172.21.15.46:0/723530467 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f932c0a6450 con 0x7f93340b0390 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.303+0000 7f9332ffd640 1 -- 172.21.15.46:0/723530467 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f932c0a67c0 con 0x7f93340b0390 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- 172.21.15.46:0/723530467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 msgr2=0x7f933413c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 --2- 172.21.15.46:0/723530467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f933413c5c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f932c009dc0 tx=0x7f932c01c820 comp rx=0 tx=0).stop 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- 172.21.15.46:0/723530467 shutdown_connections 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 --2- 172.21.15.46:0/723530467 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f933413c5c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- 172.21.15.46:0/723530467 >> 172.21.15.46:0/723530467 conn(0x7f933406a580 msgr2=0x7f933406a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- 172.21.15.46:0/723530467 shutdown_connections 2023-12-16T21:42:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7fd6497fa640 1 -- 172.21.15.46:0/3180488070 <== mon.1 v2:172.21.15.161:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fd638003c50 con 0x7fd6440338f0 2023-12-16T21:42:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- 172.21.15.46:0/723530467 wait complete. 2023-12-16T21:42:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 Processor -- start 2023-12-16T21:42:46.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7fd6497fa640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd630066470 0x7fd630068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7fd6497fa640 1 -- 172.21.15.46:0/3180488070 <== mon.1 v2:172.21.15.161:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd638082f90 con 0x7fd6440338f0 2023-12-16T21:42:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7fd650c93640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd630066470 0x7fd630068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- start start 2023-12-16T21:42:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93340af8d0 0x7f933414c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f93341e3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93340b13a0 0x7f93341e3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93341e7360 con 0x7f93340af8d0 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334141100 con 0x7f93340b13a0 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93397c8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9334141630 con 0x7f93340b0390 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93340af8d0 0x7f933414c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93340af8d0 0x7f933414c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45134/0 (socket says 172.21.15.46:45134) 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 -- 172.21.15.46:0/3988382270 learned_addr learned my addr 172.21.15.46:0/3988382270 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93337fe640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f93341e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9338fc7640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93340b13a0 0x7f93341e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 -- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 msgr2=0x7f93341e3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f93341e3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 -- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93340b13a0 msgr2=0x7f93341e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93340b13a0 0x7f93341e3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 -- 172.21.15.46:0/3988382270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932c009a70 con 0x7f93340af8d0 2023-12-16T21:42:46.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f93337fe640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f93341e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9338fc7640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93340b13a0 0x7f93341e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 -- 172.21.15.46:0/3358108641 shutdown_connections 2023-12-16T21:42:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f9333fff640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93340af8d0 0x7f933414c6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f932800c440 tx=0x7f932800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7fd650c93640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd630066470 0x7fd630068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd640009990 tx=0x7fd640009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d63fff640 1 --2- 172.21.15.46:0/1997892935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d640b3b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098a0 tx=0x7f2d5401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d816baf2183fd17 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f9330ff9640 1 -- 172.21.15.46:0/3988382270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93280052d0 con 0x7f93340af8d0 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f9330ff9640 1 -- 172.21.15.46:0/3988382270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9328005450 con 0x7f93340af8d0 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f9330ff9640 1 -- 172.21.15.46:0/3988382270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328003840 con 0x7f93340af8d0 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d62ffd640 1 -- 172.21.15.46:0/1997892935 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540ac620 con 0x7f2d640b4910 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d62ffd640 1 -- 172.21.15.46:0/1997892935 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d540ac780 con 0x7f2d640b4910 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d62ffd640 1 -- 172.21.15.46:0/1997892935 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540acb50 con 0x7f2d640b4910 2023-12-16T21:42:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/1997892935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 msgr2=0x7f2d640b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/1997892935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d640b3b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098a0 tx=0x7f2d5401c510 comp rx=0 tx=0).stop 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.307+0000 7f879e459640 1 -- 172.21.15.46:0/3358108641 wait complete. 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 Processor -- start 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 -- start start 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798071b40 0x7f879813e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 0x7f879813cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798145b20 0x7f879813d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879813fbf0 con 0x7f8798071b40 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87981ec620 con 0x7f8798145b20 2023-12-16T21:42:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879e459640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87981ec900 con 0x7f87980b05d0 2023-12-16T21:42:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/1997892935 shutdown_connections 2023-12-16T21:42:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/1997892935 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d640b3b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/1997892935 >> 172.21.15.46:0/1997892935 conn(0x7f2d640bbe40 msgr2=0x7f2d640bc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/1997892935 shutdown_connections 2023-12-16T21:42:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798071b40 0x7f879813e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798071b40 0x7f879813e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45150/0 (socket says 172.21.15.46:45150) 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 -- 172.21.15.46:0/2871339449 learned_addr learned my addr 172.21.15.46:0/2871339449 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879dc58640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798145b20 0x7f879813d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 -- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 msgr2=0x7f879813cff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 0x7f879813cff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 -- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798145b20 msgr2=0x7f879813d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798145b20 0x7f879813d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 -- 172.21.15.46:0/2871339449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879000c660 con 0x7f8798071b40 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f879d457640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798071b40 0x7f879813e840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f879400b4a0 tx=0x7f879400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93341e6d10 con 0x7f93340af8d0 2023-12-16T21:42:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9334141350 con 0x7f93340af8d0 2023-12-16T21:42:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/1997892935 wait complete. 2023-12-16T21:42:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 Processor -- start 2023-12-16T21:42:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93341e2e40 con 0x7f93340af8d0 2023-12-16T21:42:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.311+0000 7f2d6a4f2640 1 -- start start 2023-12-16T21:42:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d64142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d64140a40 0x7f2d641e6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d64146f50 0x7f2d641353c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d64141210 con 0x7f2d64140a40 2023-12-16T21:42:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d641e6f80 con 0x7f2d64146f50 2023-12-16T21:42:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d640b27b0 con 0x7f2d640b4910 2023-12-16T21:42:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f878dffb640 1 -- 172.21.15.46:0/2871339449 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794010610 con 0x7f8798071b40 2023-12-16T21:42:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d68a8f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d64146f50 0x7f2d641353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d64142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d64142b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:50664/0 (socket says 172.21.15.46:50664) 2023-12-16T21:42:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 -- 172.21.15.46:0/656705570 learned_addr learned my addr 172.21.15.46:0/656705570 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f878dffb640 1 -- 172.21.15.46:0/2871339449 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8794010790 con 0x7f8798071b40 2023-12-16T21:42:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f878dffb640 1 -- 172.21.15.46:0/2871339449 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794010b40 con 0x7f8798071b40 2023-12-16T21:42:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87981ecbe0 con 0x7f8798071b40 2023-12-16T21:42:46.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8798070d80 con 0x7f8798071b40 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 -- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d64146f50 msgr2=0x7f2d641353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d64146f50 0x7f2d641353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 -- 172.21.15.46:0/656705570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d64140a40 msgr2=0x7f2d641e6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d64140a40 0x7f2d641e6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 -- 172.21.15.46:0/656705570 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d540094d0 con 0x7f2d640b4910 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d63fff640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d64142b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098a0 tx=0x7f2d540192b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d60ff9640 1 -- 172.21.15.46:0/656705570 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540b6640 con 0x7f2d640b4910 2023-12-16T21:42:46.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d60ff9640 1 -- 172.21.15.46:0/656705570 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d540b67a0 con 0x7f2d640b4910 2023-12-16T21:42:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d641e7440 con 0x7f2d640b4910 2023-12-16T21:42:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.315+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d641e7160 con 0x7f2d640b4910 2023-12-16T21:42:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d44005260 con 0x7f2d640b4910 2023-12-16T21:42:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f2d60ff9640 1 -- 172.21.15.46:0/656705570 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d540b6b10 con 0x7f2d640b4910 2023-12-16T21:42:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8778005260 con 0x7f8798071b40 2023-12-16T21:42:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f9330ff9640 1 -- 172.21.15.46:0/3988382270 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f93280039a0 con 0x7f93340af8d0 2023-12-16T21:42:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f9330ff9640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f931c066840 0x7f931c068d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f9330ff9640 1 -- 172.21.15.46:0/3988382270 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9328083590 con 0x7f93340af8d0 2023-12-16T21:42:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f9330ff9640 1 -- 172.21.15.46:0/3988382270 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f9328054510 con 0x7f93340af8d0 2023-12-16T21:42:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f93337fe640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f931c066840 0x7f931c068d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.319+0000 7f2d60ff9640 1 -- 172.21.15.46:0/656705570 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2d540aa070 con 0x7f2d640b4910 2023-12-16T21:42:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f2d60ff9640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2d500664c0 0x7f2d50068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f2d60ff9640 1 -- 172.21.15.46:0/656705570 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d5411eb60 con 0x7f2d640b4910 2023-12-16T21:42:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7fd6497fa640 1 -- 172.21.15.46:0/3180488070 <== mon.1 v2:172.21.15.161:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fd638050530 con 0x7fd6440338f0 2023-12-16T21:42:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f93337fe640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f931c066840 0x7f931c068d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93340708d0 tx=0x7f9324009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f2d637fe640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2d500664c0 0x7f2d50068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f878dffb640 1 -- 172.21.15.46:0/2871339449 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8794010ca0 con 0x7f8798071b40 2023-12-16T21:42:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f878dffb640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f877c0664c0 0x7f877c068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f878dffb640 1 -- 172.21.15.46:0/2871339449 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8794083d70 con 0x7f8798071b40 2023-12-16T21:42:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f879cc56640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f877c0664c0 0x7f877c068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f2d637fe640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2d500664c0 0x7f2d50068980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d58000f70 tx=0x7f2d580079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.323+0000 7f879cc56640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f877c0664c0 0x7f877c068980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f879813e370 tx=0x7f8788006c30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.331+0000 7f878dffb640 1 -- 172.21.15.46:0/2871339449 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8794051410 con 0x7f8798071b40 2023-12-16T21:42:46.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.331+0000 7f2d60ff9640 1 -- 172.21.15.46:0/656705570 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2d540ec050 con 0x7f2d640b4910 2023-12-16T21:42:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.339+0000 7f2b9c9f7640 1 Processor -- start 2023-12-16T21:42:46.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.339+0000 7f2b9c9f7640 1 -- start start 2023-12-16T21:42:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b9c9f7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b90035720 0x7f2b9003a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b9c9f7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b900c4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b9c9f7640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b900398d0 0x7f2b900c4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b9c9f7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9002b0d0 con 0x7f2b90035720 2023-12-16T21:42:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b9c9f7640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b90038330 con 0x7f2b900398d0 2023-12-16T21:42:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b9c9f7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90035070 con 0x7f2b9003a8b0 2023-12-16T21:42:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b977fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b90035720 0x7f2b9003a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b977fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2b90035720 0x7f2b9003a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45162/0 (socket says 172.21.15.46:45162) 2023-12-16T21:42:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b97fff640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b900398d0 0x7f2b900c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb11a7a0640 1 Processor -- start 2023-12-16T21:42:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb11a7a0640 1 -- start start 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b97fff640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2b900398d0 0x7f2b900c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:35640/0 (socket says 172.21.15.46:35640) 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b97fff640 1 -- 172.21.15.46:0/961382233 learned_addr learned my addr 172.21.15.46:0/961382233 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b96ffd640 1 --2- 172.21.15.46:0/961382233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b900c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b977fe640 1 -- 172.21.15.46:0/961382233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 msgr2=0x7f2b900c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b977fe640 1 --2- 172.21.15.46:0/961382233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b900c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b977fe640 1 -- 172.21.15.46:0/961382233 >> v2:172.21.15.161:3300/0 conn(0x7f2b900398d0 msgr2=0x7f2b900c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b977fe640 1 --2- 172.21.15.46:0/961382233 >> v2:172.21.15.161:3300/0 conn(0x7f2b900398d0 0x7f2b900c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b977fe640 1 -- 172.21.15.46:0/961382233 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90035b00 con 0x7f2b90035720 2023-12-16T21:42:46.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7f2b97fff640 1 --2- 172.21.15.46:0/961382233 >> v2:172.21.15.161:3300/0 conn(0x7f2b900398d0 0x7f2b900c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb11a7a0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb10c0385f0 0x7fb10c034430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb11a7a0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c034970 con 0x7fb10c0c3030 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb11a7a0640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c0389d0 con 0x7fb10c0362d0 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb11a7a0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c02a060 con 0x7fb10c0385f0 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb113fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb10c0385f0 0x7fb10c034430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb113fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb10c0385f0 0x7fb10c034430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45178/0 (socket says 172.21.15.46:45178) 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb113fff640 1 -- 172.21.15.46:0/2100991832 learned_addr learned my addr 172.21.15.46:0/2100991832 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb113fff640 1 -- 172.21.15.46:0/2100991832 >> v1:172.21.15.161:6789/0 conn(0x7fb10c0362d0 legacy=0x7fb10c036760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb113fff640 1 -- 172.21.15.46:0/2100991832 >> v1:172.21.15.46:6789/0 conn(0x7fb10c0c3030 legacy=0x7fb10c0c3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.343+0000 7fb113fff640 1 -- 172.21.15.46:0/2100991832 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c02acb0 con 0x7fb10c0385f0 2023-12-16T21:42:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b977fe640 1 --2- 172.21.15.46:0/961382233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b90035720 0x7f2b9003a370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b9003fbd0 tx=0x7f2b8c01c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac824d1b0b21d8f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b967fc640 1 -- 172.21.15.46:0/961382233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c0ab600 con 0x7f2b90035720 2023-12-16T21:42:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b967fc640 1 -- 172.21.15.46:0/961382233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8c0a7050 con 0x7f2b90035720 2023-12-16T21:42:46.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b967fc640 1 -- 172.21.15.46:0/961382233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c0ab980 con 0x7f2b90035720 2023-12-16T21:42:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7fb113fff640 1 --2- 172.21.15.46:0/2100991832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0385f0 0x7fb10c034430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb100009870 tx=0x7fb10001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351f81f87a9e751f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7fb112ffd640 1 -- 172.21.15.46:0/2100991832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000ab600 con 0x7fb10c0385f0 2023-12-16T21:42:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7fb112ffd640 1 -- 172.21.15.46:0/2100991832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1000a7050 con 0x7fb10c0385f0 2023-12-16T21:42:46.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7fb112ffd640 1 -- 172.21.15.46:0/2100991832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1000ab980 con 0x7fb10c0385f0 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/961382233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b90035720 msgr2=0x7f2b9003a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/961382233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b90035720 0x7f2b9003a370 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b9003fbd0 tx=0x7f2b8c01c2d0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/961382233 shutdown_connections 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/961382233 >> v2:172.21.15.161:3300/0 conn(0x7f2b900398d0 0x7f2b900c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/961382233 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b900c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/961382233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b90035720 0x7f2b9003a370 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/961382233 >> 172.21.15.46:0/961382233 conn(0x7f2b90008660 msgr2=0x7f2b9003eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/961382233 shutdown_connections 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/961382233 wait complete. 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7fb11a7a0640 1 -- 172.21.15.46:0/2100991832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0385f0 msgr2=0x7fb10c034430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.347+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/2100991832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0385f0 0x7fb10c034430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb100009870 tx=0x7fb10001c470 comp rx=0 tx=0).stop 2023-12-16T21:42:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 Processor -- start 2023-12-16T21:42:46.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 -- start start 2023-12-16T21:42:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b90035720 0x7f2b900cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b900398d0 0x7f2b900cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b9016c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900cdeb0 con 0x7f2b900398d0 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900cbd00 con 0x7f2b90035720 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900cd1b0 con 0x7f2b9003a8b0 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7fb11a7a0640 1 -- 172.21.15.46:0/2100991832 shutdown_connections 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/2100991832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0385f0 0x7fb10c034430 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7fb11a7a0640 1 -- 172.21.15.46:0/2100991832 >> 172.21.15.46:0/2100991832 conn(0x7fb10c03a860 msgr2=0x7fb10c03ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b96ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b900398d0 0x7f2b900cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b97fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b9016c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b96ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b900398d0 0x7f2b900cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45192/0 (socket says 172.21.15.46:45192) 2023-12-16T21:42:46.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b96ffd640 1 -- 172.21.15.46:0/2497395028 learned_addr learned my addr 172.21.15.46:0/2497395028 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b977fe640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b90035720 0x7f2b900cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7fb11a7a0640 1 -- 172.21.15.46:0/2100991832 shutdown_connections 2023-12-16T21:42:46.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7fb11a7a0640 1 -- 172.21.15.46:0/2100991832 wait complete. 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b97fff640 1 -- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b90035720 msgr2=0x7f2b900cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b97fff640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b90035720 0x7f2b900cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b97fff640 1 -- 172.21.15.46:0/2497395028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b900398d0 msgr2=0x7f2b900cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b97fff640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b900398d0 0x7f2b900cb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b97fff640 1 -- 172.21.15.46:0/2497395028 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9804ed70 con 0x7f2b9003a8b0 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b96ffd640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b900398d0 0x7f2b900cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b97fff640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b9016c730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2b980797c0 tx=0x7f2b98067fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b83fff640 1 -- 172.21.15.46:0/2497395028 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9806bc60 con 0x7f2b9003a8b0 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b83fff640 1 -- 172.21.15.46:0/2497395028 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9806bdc0 con 0x7f2b9003a8b0 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c009420 con 0x7f2b9003a8b0 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b83fff640 1 -- 172.21.15.46:0/2497395028 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b98095680 con 0x7f2b9003a8b0 2023-12-16T21:42:46.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b900cd450 con 0x7f2b9003a8b0 2023-12-16T21:42:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.351+0000 7fb11a7a0640 1 Processor -- start 2023-12-16T21:42:46.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9002b530 con 0x7f2b9003a8b0 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb11a7a0640 1 -- start start 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb11a7a0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0362d0 0x7fb10c0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb11a7a0640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb10c0385f0 0x7fb10c169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb11a7a0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0c3030 0x7fb10c169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb11a7a0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c006560 con 0x7fb10c0c3030 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb11a7a0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c169280 con 0x7fb10c0362d0 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb11a7a0640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0c6480 con 0x7fb10c0385f0 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb118d3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0c3030 0x7fb10c169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb118d3d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0c3030 0x7fb10c169cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45204/0 (socket says 172.21.15.46:45204) 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb118d3d640 1 -- 172.21.15.46:0/1891239798 learned_addr learned my addr 172.21.15.46:0/1891239798 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb1137fe640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb10c0385f0 0x7fb10c169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb113fff640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0362d0 0x7fb10c0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb1137fe640 1 -- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0362d0 msgr2=0x7fb10c0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.363 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb1137fe640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0362d0 0x7fb10c0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb1137fe640 1 -- 172.21.15.46:0/1891239798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0c3030 msgr2=0x7fb10c169cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb1137fe640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0c3030 0x7fb10c169cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb1137fe640 1 -- 172.21.15.46:0/1891239798 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000094d0 con 0x7fb10c0385f0 2023-12-16T21:42:46.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7fb1137fe640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb10c0385f0 0x7fb10c169780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb114068240 tx=0x7fb114067950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7f2b83fff640 1 -- 172.21.15.46:0/2497395028 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2b980957e0 con 0x7f2b9003a8b0 2023-12-16T21:42:46.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7f2b83fff640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2b7c0664a0 0x7f2b7c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7f2b83fff640 1 -- 172.21.15.46:0/2497395028 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b98090070 con 0x7f2b9003a8b0 2023-12-16T21:42:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.355+0000 7f2b977fe640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2b7c0664a0 0x7f2b7c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.365 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.359+0000 7f2b977fe640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2b7c0664a0 0x7f2b7c068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0ad000 tx=0x7f2b8c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.367+0000 7fb110ff9640 1 -- 172.21.15.46:0/1891239798 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11406bb20 con 0x7fb10c0385f0 2023-12-16T21:42:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.367+0000 7fb110ff9640 1 -- 172.21.15.46:0/1891239798 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11406bcc0 con 0x7fb10c0385f0 2023-12-16T21:42:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.367+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c0c6760 con 0x7fb10c0385f0 2023-12-16T21:42:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.367+0000 7fb110ff9640 1 -- 172.21.15.46:0/1891239798 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114092640 con 0x7fb10c0385f0 2023-12-16T21:42:46.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.367+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c0343e0 con 0x7fb10c0385f0 2023-12-16T21:42:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.367+0000 7f2b83fff640 1 -- 172.21.15.46:0/2497395028 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2b980cb2f0 con 0x7f2b9003a8b0 2023-12-16T21:42:46.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.371+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c0053a0 con 0x7fb10c0385f0 2023-12-16T21:42:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.371+0000 7fb110ff9640 1 -- 172.21.15.46:0/1891239798 <== mon.2 v2:172.21.15.161:3301/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fb11409b3f0 con 0x7fb10c0385f0 2023-12-16T21:42:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.371+0000 7fb110ff9640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb0f8066540 0x7fb0f8068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.371+0000 7fb110ff9640 1 -- 172.21.15.46:0/1891239798 <== mon.2 v2:172.21.15.161:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1140fdd30 con 0x7fb10c0385f0 2023-12-16T21:42:46.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.371+0000 7fb113fff640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb0f8066540 0x7fb0f8068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.371+0000 7fb113fff640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb0f8066540 0x7fb0f8068a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1000ad000 tx=0x7fb1000a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.390 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.383+0000 7fb110ff9640 1 -- 172.21.15.46:0/1891239798 <== mon.2 v2:172.21.15.161:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fb1140cb2d0 con 0x7fb10c0385f0 2023-12-16T21:42:46.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.551+0000 7f924872f640 1 -- 172.21.15.46:0/3900506985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f92280057e0 con 0x7f92400b4440 2023-12-16T21:42:46.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.555+0000 7f9236ffd640 1 -- 172.21.15.46:0/3900506985 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f92380cd030 con 0x7f92400b4440 2023-12-16T21:42:46.563 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 -- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9224066540 msgr2=0x7f9224068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9224066540 0x7f9224068a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9230011fd0 tx=0x7f9230008040 comp rx=0 tx=0).stop 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 -- 172.21.15.46:0/3900506985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92400b4440 msgr2=0x7f92401eece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92400b4440 0x7f92401eece0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f923801af70 tx=0x7f9238019660 comp rx=0 tx=0).stop 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 -- 172.21.15.46:0/3900506985 shutdown_connections 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9224066540 0x7f9224068a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f92400b4440 0x7f92401eece0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f92400b2c80 0x7f92401ee7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 --2- 172.21.15.46:0/3900506985 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f92400b0310 0x7f9240158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 -- 172.21.15.46:0/3900506985 >> 172.21.15.46:0/3900506985 conn(0x7f924006a580 msgr2=0x7f9240066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 -- 172.21.15.46:0/3900506985 shutdown_connections 2023-12-16T21:42:46.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.559+0000 7f9234ff9640 1 -- 172.21.15.46:0/3900506985 wait complete. 2023-12-16T21:42:46.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-12-16T21:42:46.588 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.591+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f20bc005510 con 0x7f20d00b4300 2023-12-16T21:42:46.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.591+0000 7f20ccff9640 1 -- 172.21.15.46:0/4089399584 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20c8055740 con 0x7f20d00b4300 2023-12-16T21:42:46.598 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-16T21:42:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f20b80666c0 msgr2=0x7f20b8068b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f20b80666c0 0x7f20b8068b80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f20c0000c10 tx=0x7f20c00079d0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20d00b4300 msgr2=0x7f20d01daef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20d00b4300 0x7f20d01daef0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f20c800ccb0 tx=0x7f20c8007700 comp rx=0 tx=0).stop 2023-12-16T21:42:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 shutdown_connections 2023-12-16T21:42:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f20b80666c0 0x7f20b8068b80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20d00b4f30 0x7f20d01db430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20d00b4300 0x7f20d01daef0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 --2- 172.21.15.46:0/4089399584 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f20d00aec40 0x7f20d0070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.602 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 >> 172.21.15.46:0/4089399584 conn(0x7f20d006a580 msgr2=0x7f20d0066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 shutdown_connections 2023-12-16T21:42:46.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.595+0000 7f20d6592640 1 -- 172.21.15.46:0/4089399584 wait complete. 2023-12-16T21:42:46.624 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-12-16T21:42:46.624 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.627+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd6440c6220 con 0x7fd6440338f0 2023-12-16T21:42:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.627+0000 7fd6497fa640 1 -- 172.21.15.46:0/3180488070 <== mon.1 v2:172.21.15.161:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd638017020 con 0x7fd6440338f0 2023-12-16T21:42:46.634 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd630066470 msgr2=0x7fd630068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd630066470 0x7fd630068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd640009990 tx=0x7fd640009110 comp rx=0 tx=0).stop 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440338f0 msgr2=0x7fd6440c1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440338f0 0x7fd6440c1010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd63800c390 tx=0x7fd63800c790 comp rx=0 tx=0).stop 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 shutdown_connections 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd630066470 0x7fd630068930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6440c85d0 0x7fd644168dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd6440338f0 0x7fd6440c1010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 --2- 172.21.15.46:0/3180488070 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd644006e80 0x7fd6440c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 >> 172.21.15.46:0/3180488070 conn(0x7fd64403a830 msgr2=0x7fd6440c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 shutdown_connections 2023-12-16T21:42:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7fd651c95640 1 -- 172.21.15.46:0/3180488070 wait complete. 2023-12-16T21:42:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2b900cd640 con 0x7f2b9003a8b0 2023-12-16T21:42:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7f2b83fff640 1 -- 172.21.15.46:0/2497395028 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b980ce910 con 0x7f2b9003a8b0 2023-12-16T21:42:46.640 INFO:teuthology.orchestra.run.smithi046.stdout:55834574853 2023-12-16T21:42:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.631+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f87780057e0 con 0x7f8798071b40 2023-12-16T21:42:46.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f878dffb640 1 -- 172.21.15.46:0/2871339449 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f879401e020 con 0x7f8798071b40 2023-12-16T21:42:46.640 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-16T21:42:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2b7c0664a0 msgr2=0x7f2b7c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2b7c0664a0 0x7f2b7c068960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0ad000 tx=0x7f2b8c0a6000 comp rx=0 tx=0).stop 2023-12-16T21:42:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 msgr2=0x7f2b9016c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b9016c730 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2b980797c0 tx=0x7f2b98067fa0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 shutdown_connections 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2b7c0664a0 0x7f2b7c068960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2b9003a8b0 0x7f2b9016c730 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2b900398d0 0x7f2b900cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 --2- 172.21.15.46:0/2497395028 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2b90035720 0x7f2b900cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 >> 172.21.15.46:0/2497395028 conn(0x7f2b90008660 msgr2=0x7f2b900c2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 shutdown_connections 2023-12-16T21:42:46.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f2b9c9f7640 1 -- 172.21.15.46:0/2497395028 wait complete. 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f877c0664c0 msgr2=0x7f877c068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f877c0664c0 0x7f877c068980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f879813e370 tx=0x7f8788006c30 comp rx=0 tx=0).stop 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798071b40 msgr2=0x7f879813e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798071b40 0x7f879813e840 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f879400b4a0 tx=0x7f879400b960 comp rx=0 tx=0).stop 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 shutdown_connections 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f877c0664c0 0x7f877c068980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8798145b20 0x7f879813d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87980b05d0 0x7f879813cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 --2- 172.21.15.46:0/2871339449 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8798071b40 0x7f879813e840 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 >> 172.21.15.46:0/2871339449 conn(0x7f87980b7ca0 msgr2=0x7f879813c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 shutdown_connections 2023-12-16T21:42:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.635+0000 7f879e459640 1 -- 172.21.15.46:0/2871339449 wait complete. 2023-12-16T21:42:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.639+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93340688f0 con 0x7f93340af8d0 2023-12-16T21:42:46.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f9330ff9640 1 -- 172.21.15.46:0/3988382270 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f93280540a0 con 0x7f93340af8d0 2023-12-16T21:42:46.649 INFO:teuthology.orchestra.run.smithi046.stdout:55834574855 2023-12-16T21:42:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f931c066840 msgr2=0x7f931c068d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f931c066840 0x7f931c068d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f93340708d0 tx=0x7f9324009040 comp rx=0 tx=0).stop 2023-12-16T21:42:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93340af8d0 msgr2=0x7f933414c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93340af8d0 0x7f933414c6d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f932800c440 tx=0x7f932800c900 comp rx=0 tx=0).stop 2023-12-16T21:42:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 shutdown_connections 2023-12-16T21:42:46.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f931c066840 0x7f931c068d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f93340b13a0 0x7f93341e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93340b0390 0x7f93341e3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 --2- 172.21.15.46:0/3988382270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93340af8d0 0x7f933414c6d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 >> 172.21.15.46:0/3988382270 conn(0x7f933406a580 msgr2=0x7f9334071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.643+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 shutdown_connections 2023-12-16T21:42:46.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.647+0000 7f93397c8640 1 -- 172.21.15.46:0/3988382270 wait complete. 2023-12-16T21:42:46.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.655+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f2d440057e0 con 0x7f2d640b4910 2023-12-16T21:42:46.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-12-16T21:42:46.665 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2023-12-16T21:42:46.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.655+0000 7f2d60ff9640 1 -- 172.21.15.46:0/656705570 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d540ef670 con 0x7f2d640b4910 2023-12-16T21:42:46.666 INFO:teuthology.orchestra.run.smithi046.stdout:55834574854 2023-12-16T21:42:46.666 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-12-16T21:42:46.666 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2d500664c0 msgr2=0x7f2d50068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2d500664c0 0x7f2d50068980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2d58000f70 tx=0x7f2d580079c0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 msgr2=0x7f2d64142b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d64142b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2d540098a0 tx=0x7f2d540192b0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 shutdown_connections 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2d500664c0 0x7f2d50068980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2d64146f50 0x7f2d641353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d64140a40 0x7f2d641e6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 --2- 172.21.15.46:0/656705570 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2d640b4910 0x7f2d64142b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 >> 172.21.15.46:0/656705570 conn(0x7f2d640bbe40 msgr2=0x7f2d640b6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 shutdown_connections 2023-12-16T21:42:46.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.659+0000 7f2d6a4f2640 1 -- 172.21.15.46:0/656705570 wait complete. 2023-12-16T21:42:46.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-12-16T21:42:46.676 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-12-16T21:42:46.690 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.691+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fb10c0c3610 con 0x7fb10c0385f0 2023-12-16T21:42:46.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.691+0000 7fb110ff9640 1 -- 172.21.15.46:0/1891239798 <== mon.2 v2:172.21.15.161:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1140ce8f0 con 0x7fb10c0385f0 2023-12-16T21:42:46.699 INFO:teuthology.orchestra.run.smithi046.stdout:55834574856 2023-12-16T21:42:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb0f8066540 msgr2=0x7fb0f8068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb0f8066540 0x7fb0f8068a00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1000ad000 tx=0x7fb1000a6000 comp rx=0 tx=0).stop 2023-12-16T21:42:46.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb10c0385f0 msgr2=0x7fb10c169780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb10c0385f0 0x7fb10c169780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb114068240 tx=0x7fb114067950 comp rx=0 tx=0).stop 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 shutdown_connections 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb0f8066540 0x7fb0f8068a00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb10c0c3030 0x7fb10c169cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb10c0385f0 0x7fb10c169780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 --2- 172.21.15.46:0/1891239798 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb10c0362d0 0x7fb10c0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 >> 172.21.15.46:0/1891239798 conn(0x7fb10c03a860 msgr2=0x7fb10c038bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 shutdown_connections 2023-12-16T21:42:46.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.695+0000 7fb11a7a0640 1 -- 172.21.15.46:0/1891239798 wait complete. 2023-12-16T21:42:46.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-12-16T21:42:46.716 DEBUG:teuthology.parallel:result is None 2023-12-16T21:42:46.716 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-16T21:42:46.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:42:46.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.859+0000 7fa4d8bc7640 1 Processor -- start 2023-12-16T21:42:46.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.859+0000 7fa4d8bc7640 1 -- start start 2023-12-16T21:42:46.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4d4071540 0x7fa4d40b5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d40b72a0 con 0x7fa4d4140550 2023-12-16T21:42:46.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d406c090 con 0x7fa4d40b6ec0 2023-12-16T21:42:46.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d406be20 con 0x7fa4d4071540 2023-12-16T21:42:46.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4d4071540 0x7fa4d40b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fa4d4071540 0x7fa4d40b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45216/0 (socket says 172.21.15.46:45216) 2023-12-16T21:42:46.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d1d9b640 1 -- 172.21.15.46:0/2024800944 learned_addr learned my addr 172.21.15.46:0/2024800944 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d1d9b640 1 -- 172.21.15.46:0/2024800944 >> v1:172.21.15.161:6789/0 conn(0x7fa4d40b6ec0 legacy=0x7fa4d40b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d1d9b640 1 -- 172.21.15.46:0/2024800944 >> v1:172.21.15.46:6789/0 conn(0x7fa4d4140550 legacy=0x7fa4d4140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:46.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d1d9b640 1 -- 172.21.15.46:0/2024800944 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d4070a60 con 0x7fa4d4071540 2023-12-16T21:42:46.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d1d9b640 1 --2- 172.21.15.46:0/2024800944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4071540 0x7fa4d40b5540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009870 tx=0x7fa4c801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4633c118c6fab7e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d159a640 1 -- 172.21.15.46:0/2024800944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c80ad680 con 0x7fa4d4071540 2023-12-16T21:42:46.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d159a640 1 -- 172.21.15.46:0/2024800944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c80ad7e0 con 0x7fa4d4071540 2023-12-16T21:42:46.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d159a640 1 -- 172.21.15.46:0/2024800944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c80adb70 con 0x7fa4d4071540 2023-12-16T21:42:46.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/2024800944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4071540 msgr2=0x7fa4d40b5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/2024800944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4071540 0x7fa4d40b5540 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009870 tx=0x7fa4c801c470 comp rx=0 tx=0).stop 2023-12-16T21:42:46.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/2024800944 shutdown_connections 2023-12-16T21:42:46.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/2024800944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4071540 0x7fa4d40b5540 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/2024800944 >> 172.21.15.46:0/2024800944 conn(0x7fa4d406a980 msgr2=0x7fa4d406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:46.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.863+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/2024800944 shutdown_connections 2023-12-16T21:42:46.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/2024800944 wait complete. 2023-12-16T21:42:46.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 Processor -- start 2023-12-16T21:42:46.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- start start 2023-12-16T21:42:46.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d4071540 0x7fa4d413f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4d40b6ec0 0x7fa4d41478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4140550 0x7fa4d4148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d413e390 con 0x7fa4d4140550 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d4147e10 con 0x7fa4d4071540 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d4148660 con 0x7fa4d40b6ec0 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4140550 0x7fa4d4148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4140550 0x7fa4d4148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45226/0 (socket says 172.21.15.46:45226) 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 -- 172.21.15.46:0/200688630 learned_addr learned my addr 172.21.15.46:0/200688630 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d259c640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d4071540 0x7fa4d413f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d1d9b640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4d40b6ec0 0x7fa4d41478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 -- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4d40b6ec0 msgr2=0x7fa4d41478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4d40b6ec0 0x7fa4d41478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 -- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d4071540 msgr2=0x7fa4d413f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d4071540 0x7fa4d413f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 -- 172.21.15.46:0/200688630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c80094d0 con 0x7fa4d4140550 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d2d9d640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4140550 0x7fa4d4148af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00c760 tx=0x7fa4cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4c2ffd640 1 -- 172.21.15.46:0/200688630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc003b80 con 0x7fa4d4140550 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4c2ffd640 1 -- 172.21.15.46:0/200688630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc003ce0 con 0x7fa4d4140550 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d4148010 con 0x7fa4d4140550 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4c2ffd640 1 -- 172.21.15.46:0/200688630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc00aa90 con 0x7fa4d4140550 2023-12-16T21:42:46.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d413ff60 con 0x7fa4d4140550 2023-12-16T21:42:46.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.867+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d4066db0 con 0x7fa4d4140550 2023-12-16T21:42:46.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.871+0000 7fa4c2ffd640 1 -- 172.21.15.46:0/200688630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fa4cc00abf0 con 0x7fa4d4140550 2023-12-16T21:42:46.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.871+0000 7fa4c2ffd640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa4b4066470 0x7fa4b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:46.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.871+0000 7fa4c2ffd640 1 -- 172.21.15.46:0/200688630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4cc083b70 con 0x7fa4d4140550 2023-12-16T21:42:46.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.871+0000 7fa4d259c640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa4b4066470 0x7fa4b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:46.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.871+0000 7fa4d259c640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa4b4066470 0x7fa4b4068930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40016c0 tx=0x7fa4c4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:46.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:46.875+0000 7fa4c2ffd640 1 -- 172.21.15.46:0/200688630 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fa4cc050ff0 con 0x7fa4d4140550 2023-12-16T21:42:47.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.023+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4d40688f0 con 0x7fa4b4066470 2023-12-16T21:42:47.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.047+0000 7fa4c2ffd640 1 -- 172.21.15.46:0/200688630 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264573 (secure 0 0 0) 0x7fa4d40688f0 con 0x7fa4b4066470 2023-12-16T21:42:47.056 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:47.058 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa4b4066470 msgr2=0x7fa4b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa4b4066470 0x7fa4b4068930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa4c40016c0 tx=0x7fa4c4009040 comp rx=0 tx=0).stop 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4140550 msgr2=0x7fa4d4148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4140550 0x7fa4d4148af0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00c760 tx=0x7fa4cc00cc20 comp rx=0 tx=0).stop 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 shutdown_connections 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa4b4066470 0x7fa4b4068930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa4d4140550 0x7fa4d4148af0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa4d40b6ec0 0x7fa4d41478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 --2- 172.21.15.46:0/200688630 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa4d4071540 0x7fa4d413f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 >> 172.21.15.46:0/200688630 conn(0x7fa4d406a980 msgr2=0x7fa4d4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 shutdown_connections 2023-12-16T21:42:47.059 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.051+0000 7fa4d8bc7640 1 -- 172.21.15.46:0/200688630 wait complete. 2023-12-16T21:42:47.073 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-16T21:42:46.352226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333636+0000","last_change":"2023-12-16T21:42:41.368574+0000","last_active":"2023-12-16T21:42:42.333636+0000","last_peered":"2023-12-16T21:42:42.333636+0000","last_clean":"2023-12-16T21:42:42.333636+0000","last_became_active":"2023-12-16T21:42:41.367555+0000","last_became_peered":"2023-12-16T21:42:41.367555+0000","last_unstale":"2023-12-16T21:42:42.333636+0000","last_undegraded":"2023-12-16T21:42:42.333636+0000","last_fullsized":"2023-12-16T21:42:42.333636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329805+0000","last_change":"2023-12-16T21:42:41.344487+0000","last_active":"2023-12-16T21:42:42.329805+0000","last_peered":"2023-12-16T21:42:42.329805+0000","last_clean":"2023-12-16T21:42:42.329805+0000","last_became_active":"2023-12-16T21:42:41.343798+0000","last_became_peered":"2023-12-16T21:42:41.343798+0000","last_unstale":"2023-12-16T21:42:42.329805+0000","last_undegraded":"2023-12-16T21:42:42.329805+0000","last_fullsized":"2023-12-16T21:42:42.329805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471802+0000","last_change":"2023-12-16T21:42:41.364221+0000","last_active":"2023-12-16T21:42:42.471802+0000","last_peered":"2023-12-16T21:42:42.471802+0000","last_clean":"2023-12-16T21:42:42.471802+0000","last_became_active":"2023-12-16T21:42:41.362506+0000","last_became_peered":"2023-12-16T21:42:41.362506+0000","last_unstale":"2023-12-16T21:42:42.471802+0000","last_undegraded":"2023-12-16T21:42:42.471802+0000","last_fullsized":"2023-12-16T21:42:42.471802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474631+0000","last_change":"2023-12-16T21:42:41.384692+0000","last_active":"2023-12-16T21:42:42.474631+0000","last_peered":"2023-12-16T21:42:42.474631+0000","last_clean":"2023-12-16T21:42:42.474631+0000","last_became_active":"2023-12-16T21:42:41.384156+0000","last_became_peered":"2023-12-16T21:42:41.384156+0000","last_unstale":"2023-12-16T21:42:42.474631+0000","last_undegraded":"2023-12-16T21:42:42.474631+0000","last_fullsized":"2023-12-16T21:42:42.474631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333576+0000","last_change":"2023-12-16T21:42:41.355491+0000","last_active":"2023-12-16T21:42:42.333576+0000","last_peered":"2023-12-16T21:42:42.333576+0000","last_clean":"2023-12-16T21:42:42.333576+0000","last_became_active":"2023-12-16T21:42:41.354519+0000","last_became_peered":"2023-12-16T21:42:41.354519+0000","last_unstale":"2023-12-16T21:42:42.333576+0000","last_undegraded":"2023-12-16T21:42:42.333576+0000","last_fullsized":"2023-12-16T21:42:42.333576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474628+0000","last_change":"2023-12-16T21:42:41.367454+0000","last_active":"2023-12-16T21:42:42.474628+0000","last_peered":"2023-12-16T21:42:42.474628+0000","last_clean":"2023-12-16T21:42:42.474628+0000","last_became_active":"2023-12-16T21:42:41.366663+0000","last_became_peered":"2023-12-16T21:42:41.366663+0000","last_unstale":"2023-12-16T21:42:42.474628+0000","last_undegraded":"2023-12-16T21:42:42.474628+0000","last_fullsized":"2023-12-16T21:42:42.474628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332459+0000","last_change":"2023-12-16T21:42:41.375370+0000","last_active":"2023-12-16T21:42:42.332459+0000","last_peered":"2023-12-16T21:42:42.332459+0000","last_clean":"2023-12-16T21:42:42.332459+0000","last_became_active":"2023-12-16T21:42:41.374733+0000","last_became_peered":"2023-12-16T21:42:41.374733+0000","last_unstale":"2023-12-16T21:42:42.332459+0000","last_undegraded":"2023-12-16T21:42:42.332459+0000","last_fullsized":"2023-12-16T21:42:42.332459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471989+0000","last_change":"2023-12-16T21:42:41.385909+0000","last_active":"2023-12-16T21:42:42.471989+0000","last_peered":"2023-12-16T21:42:42.471989+0000","last_clean":"2023-12-16T21:42:42.471989+0000","last_became_active":"2023-12-16T21:42:41.385386+0000","last_became_peered":"2023-12-16T21:42:41.385386+0000","last_unstale":"2023-12-16T21:42:42.471989+0000","last_undegraded":"2023-12-16T21:42:42.471989+0000","last_fullsized":"2023-12-16T21:42:42.471989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333967+0000","last_change":"2023-12-16T21:42:41.374260+0000","last_active":"2023-12-16T21:42:42.333967+0000","last_peered":"2023-12-16T21:42:42.333967+0000","last_clean":"2023-12-16T21:42:42.333967+0000","last_became_active":"2023-12-16T21:42:41.373101+0000","last_became_peered":"2023-12-16T21:42:41.373101+0000","last_unstale":"2023-12-16T21:42:42.333967+0000","last_undegraded":"2023-12-16T21:42:42.333967+0000","last_fullsized":"2023-12-16T21:42:42.333967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475842+0000","last_change":"2023-12-16T21:42:41.386312+0000","last_active":"2023-12-16T21:42:42.475842+0000","last_peered":"2023-12-16T21:42:42.475842+0000","last_clean":"2023-12-16T21:42:42.475842+0000","last_became_active":"2023-12-16T21:42:41.385651+0000","last_became_peered":"2023-12-16T21:42:41.385651+0000","last_unstale":"2023-12-16T21:42:42.475842+0000","last_undegraded":"2023-12-16T21:42:42.475842+0000","last_fullsized":"2023-12-16T21:42:42.475842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475260+0000","last_change":"2023-12-16T21:42:41.385382+0000","last_active":"2023-12-16T21:42:42.475260+0000","last_peered":"2023-12-16T21:42:42.475260+0000","last_clean":"2023-12-16T21:42:42.475260+0000","last_became_active":"2023-12-16T21:42:41.384671+0000","last_became_peered":"2023-12-16T21:42:41.384671+0000","last_unstale":"2023-12-16T21:42:42.475260+0000","last_undegraded":"2023-12-16T21:42:42.475260+0000","last_fullsized":"2023-12-16T21:42:42.475260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335258+0000","last_change":"2023-12-16T21:42:41.373439+0000","last_active":"2023-12-16T21:42:42.335258+0000","last_peered":"2023-12-16T21:42:42.335258+0000","last_clean":"2023-12-16T21:42:42.335258+0000","last_became_active":"2023-12-16T21:42:41.372462+0000","last_became_peered":"2023-12-16T21:42:41.372462+0000","last_unstale":"2023-12-16T21:42:42.335258+0000","last_undegraded":"2023-12-16T21:42:42.335258+0000","last_fullsized":"2023-12-16T21:42:42.335258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471349+0000","last_change":"2023-12-16T21:42:41.366622+0000","last_active":"2023-12-16T21:42:42.471349+0000","last_peered":"2023-12-16T21:42:42.471349+0000","last_clean":"2023-12-16T21:42:42.471349+0000","last_became_active":"2023-12-16T21:42:41.365732+0000","last_became_peered":"2023-12-16T21:42:41.365732+0000","last_unstale":"2023-12-16T21:42:42.471349+0000","last_undegraded":"2023-12-16T21:42:42.471349+0000","last_fullsized":"2023-12-16T21:42:42.471349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472312+0000","last_change":"2023-12-16T21:42:41.347792+0000","last_active":"2023-12-16T21:42:42.472312+0000","last_peered":"2023-12-16T21:42:42.472312+0000","last_clean":"2023-12-16T21:42:42.472312+0000","last_became_active":"2023-12-16T21:42:41.345588+0000","last_became_peered":"2023-12-16T21:42:41.345588+0000","last_unstale":"2023-12-16T21:42:42.472312+0000","last_undegraded":"2023-12-16T21:42:42.472312+0000","last_fullsized":"2023-12-16T21:42:42.472312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329364+0000","last_change":"2023-12-16T21:42:41.343823+0000","last_active":"2023-12-16T21:42:42.329364+0000","last_peered":"2023-12-16T21:42:42.329364+0000","last_clean":"2023-12-16T21:42:42.329364+0000","last_became_active":"2023-12-16T21:42:41.343022+0000","last_became_peered":"2023-12-16T21:42:41.343022+0000","last_unstale":"2023-12-16T21:42:42.329364+0000","last_undegraded":"2023-12-16T21:42:42.329364+0000","last_fullsized":"2023-12-16T21:42:42.329364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474121+0000","last_change":"2023-12-16T21:42:41.383653+0000","last_active":"2023-12-16T21:42:42.474121+0000","last_peered":"2023-12-16T21:42:42.474121+0000","last_clean":"2023-12-16T21:42:42.474121+0000","last_became_active":"2023-12-16T21:42:41.383223+0000","last_became_peered":"2023-12-16T21:42:41.383223+0000","last_unstale":"2023-12-16T21:42:42.474121+0000","last_undegraded":"2023-12-16T21:42:42.474121+0000","last_fullsized":"2023-12-16T21:42:42.474121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476620+0000","last_change":"2023-12-16T21:42:41.366575+0000","last_active":"2023-12-16T21:42:42.476620+0000","last_peered":"2023-12-16T21:42:42.476620+0000","last_clean":"2023-12-16T21:42:42.476620+0000","last_became_active":"2023-12-16T21:42:41.365758+0000","last_became_peered":"2023-12-16T21:42:41.365758+0000","last_unstale":"2023-12-16T21:42:42.476620+0000","last_undegraded":"2023-12-16T21:42:42.476620+0000","last_fullsized":"2023-12-16T21:42:42.476620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471210+0000","last_change":"2023-12-16T21:42:41.371042+0000","last_active":"2023-12-16T21:42:42.471210+0000","last_peered":"2023-12-16T21:42:42.471210+0000","last_clean":"2023-12-16T21:42:42.471210+0000","last_became_active":"2023-12-16T21:42:41.370204+0000","last_became_peered":"2023-12-16T21:42:41.370204+0000","last_unstale":"2023-12-16T21:42:42.471210+0000","last_undegraded":"2023-12-16T21:42:42.471210+0000","last_fullsized":"2023-12-16T21:42:42.471210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335291+0000","last_change":"2023-12-16T21:42:41.360173+0000","last_active":"2023-12-16T21:42:42.335291+0000","last_peered":"2023-12-16T21:42:42.335291+0000","last_clean":"2023-12-16T21:42:42.335291+0000","last_became_active":"2023-12-16T21:42:41.359373+0000","last_became_peered":"2023-12-16T21:42:41.359373+0000","last_unstale":"2023-12-16T21:42:42.335291+0000","last_undegraded":"2023-12-16T21:42:42.335291+0000","last_fullsized":"2023-12-16T21:42:42.335291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:59:44.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475288+0000","last_change":"2023-12-16T21:42:41.365867+0000","last_active":"2023-12-16T21:42:42.475288+0000","last_peered":"2023-12-16T21:42:42.475288+0000","last_clean":"2023-12-16T21:42:42.475288+0000","last_became_active":"2023-12-16T21:42:41.365202+0000","last_became_peered":"2023-12-16T21:42:41.365202+0000","last_unstale":"2023-12-16T21:42:42.475288+0000","last_undegraded":"2023-12-16T21:42:42.475288+0000","last_fullsized":"2023-12-16T21:42:42.475288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471522+0000","last_change":"2023-12-16T21:42:41.377468+0000","last_active":"2023-12-16T21:42:42.471522+0000","last_peered":"2023-12-16T21:42:42.471522+0000","last_clean":"2023-12-16T21:42:42.471522+0000","last_became_active":"2023-12-16T21:42:41.376856+0000","last_became_peered":"2023-12-16T21:42:41.376856+0000","last_unstale":"2023-12-16T21:42:42.471522+0000","last_undegraded":"2023-12-16T21:42:42.471522+0000","last_fullsized":"2023-12-16T21:42:42.471522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:55.715961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475667+0000","last_change":"2023-12-16T21:42:41.366088+0000","last_active":"2023-12-16T21:42:42.475667+0000","last_peered":"2023-12-16T21:42:42.475667+0000","last_clean":"2023-12-16T21:42:42.475667+0000","last_became_active":"2023-12-16T21:42:41.365380+0000","last_became_peered":"2023-12-16T21:42:41.365380+0000","last_unstale":"2023-12-16T21:42:42.475667+0000","last_undegraded":"2023-12-16T21:42:42.475667+0000","last_fullsized":"2023-12-16T21:42:42.475667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471681+0000","last_change":"2023-12-16T21:42:41.378284+0000","last_active":"2023-12-16T21:42:42.471681+0000","last_peered":"2023-12-16T21:42:42.471681+0000","last_clean":"2023-12-16T21:42:42.471681+0000","last_became_active":"2023-12-16T21:42:41.377070+0000","last_became_peered":"2023-12-16T21:42:41.377070+0000","last_unstale":"2023-12-16T21:42:42.471681+0000","last_undegraded":"2023-12-16T21:42:42.471681+0000","last_fullsized":"2023-12-16T21:42:42.471681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475272+0000","last_change":"2023-12-16T21:42:41.376780+0000","last_active":"2023-12-16T21:42:42.475272+0000","last_peered":"2023-12-16T21:42:42.475272+0000","last_clean":"2023-12-16T21:42:42.475272+0000","last_became_active":"2023-12-16T21:42:41.376224+0000","last_became_peered":"2023-12-16T21:42:41.376224+0000","last_unstale":"2023-12-16T21:42:42.475272+0000","last_undegraded":"2023-12-16T21:42:42.475272+0000","last_fullsized":"2023-12-16T21:42:42.475272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472816+0000","last_change":"2023-12-16T21:42:41.372281+0000","last_active":"2023-12-16T21:42:42.472816+0000","last_peered":"2023-12-16T21:42:42.472816+0000","last_clean":"2023-12-16T21:42:42.472816+0000","last_became_active":"2023-12-16T21:42:41.371551+0000","last_became_peered":"2023-12-16T21:42:41.371551+0000","last_unstale":"2023-12-16T21:42:42.472816+0000","last_undegraded":"2023-12-16T21:42:42.472816+0000","last_fullsized":"2023-12-16T21:42:42.472816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475958+0000","last_change":"2023-12-16T21:42:41.375672+0000","last_active":"2023-12-16T21:42:42.475958+0000","last_peered":"2023-12-16T21:42:42.475958+0000","last_clean":"2023-12-16T21:42:42.475958+0000","last_became_active":"2023-12-16T21:42:41.370826+0000","last_became_peered":"2023-12-16T21:42:41.370826+0000","last_unstale":"2023-12-16T21:42:42.475958+0000","last_undegraded":"2023-12-16T21:42:42.475958+0000","last_fullsized":"2023-12-16T21:42:42.475958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329872+0000","last_change":"2023-12-16T21:42:41.368001+0000","last_active":"2023-12-16T21:42:42.329872+0000","last_peered":"2023-12-16T21:42:42.329872+0000","last_clean":"2023-12-16T21:42:42.329872+0000","last_became_active":"2023-12-16T21:42:41.367168+0000","last_became_peered":"2023-12-16T21:42:41.367168+0000","last_unstale":"2023-12-16T21:42:42.329872+0000","last_undegraded":"2023-12-16T21:42:42.329872+0000","last_fullsized":"2023-12-16T21:42:42.329872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472910+0000","last_change":"2023-12-16T21:42:41.377845+0000","last_active":"2023-12-16T21:42:42.472910+0000","last_peered":"2023-12-16T21:42:42.472910+0000","last_clean":"2023-12-16T21:42:42.472910+0000","last_became_active":"2023-12-16T21:42:41.377023+0000","last_became_peered":"2023-12-16T21:42:41.377023+0000","last_unstale":"2023-12-16T21:42:42.472910+0000","last_undegraded":"2023-12-16T21:42:42.472910+0000","last_fullsized":"2023-12-16T21:42:42.472910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472206+0000","last_change":"2023-12-16T21:42:41.371637+0000","last_active":"2023-12-16T21:42:42.472206+0000","last_peered":"2023-12-16T21:42:42.472206+0000","last_clean":"2023-12-16T21:42:42.472206+0000","last_became_active":"2023-12-16T21:42:41.370853+0000","last_became_peered":"2023-12-16T21:42:41.370853+0000","last_unstale":"2023-12-16T21:42:42.472206+0000","last_undegraded":"2023-12-16T21:42:42.472206+0000","last_fullsized":"2023-12-16T21:42:42.472206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475286+0000","last_change":"2023-12-16T21:42:41.375235+0000","last_active":"2023-12-16T21:42:42.475286+0000","last_peered":"2023-12-16T21:42:42.475286+0000","last_clean":"2023-12-16T21:42:42.475286+0000","last_became_active":"2023-12-16T21:42:41.374570+0000","last_became_peered":"2023-12-16T21:42:41.374570+0000","last_unstale":"2023-12-16T21:42:42.475286+0000","last_undegraded":"2023-12-16T21:42:42.475286+0000","last_fullsized":"2023-12-16T21:42:42.475286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472367+0000","last_change":"2023-12-16T21:42:41.383749+0000","last_active":"2023-12-16T21:42:42.472367+0000","last_peered":"2023-12-16T21:42:42.472367+0000","last_clean":"2023-12-16T21:42:42.472367+0000","last_became_active":"2023-12-16T21:42:41.383009+0000","last_became_peered":"2023-12-16T21:42:41.383009+0000","last_unstale":"2023-12-16T21:42:42.472367+0000","last_undegraded":"2023-12-16T21:42:42.472367+0000","last_fullsized":"2023-12-16T21:42:42.472367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473979+0000","last_change":"2023-12-16T21:42:41.379801+0000","last_active":"2023-12-16T21:42:42.473979+0000","last_peered":"2023-12-16T21:42:42.473979+0000","last_clean":"2023-12-16T21:42:42.473979+0000","last_became_active":"2023-12-16T21:42:41.379000+0000","last_became_peered":"2023-12-16T21:42:41.379000+0000","last_unstale":"2023-12-16T21:42:42.473979+0000","last_undegraded":"2023-12-16T21:42:42.473979+0000","last_fullsized":"2023-12-16T21:42:42.473979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333784+0000","last_change":"2023-12-16T21:42:41.351207+0000","last_active":"2023-12-16T21:42:42.333784+0000","last_peered":"2023-12-16T21:42:42.333784+0000","last_clean":"2023-12-16T21:42:42.333784+0000","last_became_active":"2023-12-16T21:42:41.349156+0000","last_became_peered":"2023-12-16T21:42:41.349156+0000","last_unstale":"2023-12-16T21:42:42.333784+0000","last_undegraded":"2023-12-16T21:42:42.333784+0000","last_fullsized":"2023-12-16T21:42:42.333784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:52.130538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472650+0000","last_change":"2023-12-16T21:42:40.442950+0000","last_active":"2023-12-16T21:42:42.472650+0000","last_peered":"2023-12-16T21:42:42.472650+0000","last_clean":"2023-12-16T21:42:42.472650+0000","last_became_active":"2023-12-16T21:42:40.442532+0000","last_became_peered":"2023-12-16T21:42:40.442532+0000","last_unstale":"2023-12-16T21:42:42.472650+0000","last_undegraded":"2023-12-16T21:42:42.472650+0000","last_fullsized":"2023-12-16T21:42:42.472650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472291+0000","last_change":"2023-12-16T21:42:41.384406+0000","last_active":"2023-12-16T21:42:42.472291+0000","last_peered":"2023-12-16T21:42:42.472291+0000","last_clean":"2023-12-16T21:42:42.472291+0000","last_became_active":"2023-12-16T21:42:41.383814+0000","last_became_peered":"2023-12-16T21:42:41.383814+0000","last_unstale":"2023-12-16T21:42:42.472291+0000","last_undegraded":"2023-12-16T21:42:42.472291+0000","last_fullsized":"2023-12-16T21:42:42.472291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475081+0000","last_change":"2023-12-16T21:42:40.437381+0000","last_active":"2023-12-16T21:42:42.475081+0000","last_peered":"2023-12-16T21:42:42.475081+0000","last_clean":"2023-12-16T21:42:42.475081+0000","last_became_active":"2023-12-16T21:42:40.436469+0000","last_became_peered":"2023-12-16T21:42:40.436469+0000","last_unstale":"2023-12-16T21:42:42.475081+0000","last_undegraded":"2023-12-16T21:42:42.475081+0000","last_fullsized":"2023-12-16T21:42:42.475081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471777+0000","last_change":"2023-12-16T21:42:40.432572+0000","last_active":"2023-12-16T21:42:42.471777+0000","last_peered":"2023-12-16T21:42:42.471777+0000","last_clean":"2023-12-16T21:42:42.471777+0000","last_became_active":"2023-12-16T21:42:40.431973+0000","last_became_peered":"2023-12-16T21:42:40.431973+0000","last_unstale":"2023-12-16T21:42:42.471777+0000","last_undegraded":"2023-12-16T21:42:42.471777+0000","last_fullsized":"2023-12-16T21:42:42.471777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472032+0000","last_change":"2023-12-16T21:42:41.372173+0000","last_active":"2023-12-16T21:42:42.472032+0000","last_peered":"2023-12-16T21:42:42.472032+0000","last_clean":"2023-12-16T21:42:42.472032+0000","last_became_active":"2023-12-16T21:42:41.371362+0000","last_became_peered":"2023-12-16T21:42:41.371362+0000","last_unstale":"2023-12-16T21:42:42.472032+0000","last_undegraded":"2023-12-16T21:42:42.472032+0000","last_fullsized":"2023-12-16T21:42:42.472032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474354+0000","last_change":"2023-12-16T21:42:40.458568+0000","last_active":"2023-12-16T21:42:42.474354+0000","last_peered":"2023-12-16T21:42:42.474354+0000","last_clean":"2023-12-16T21:42:42.474354+0000","last_became_active":"2023-12-16T21:42:40.457780+0000","last_became_peered":"2023-12-16T21:42:40.457780+0000","last_unstale":"2023-12-16T21:42:42.474354+0000","last_undegraded":"2023-12-16T21:42:42.474354+0000","last_fullsized":"2023-12-16T21:42:42.474354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329203+0000","last_change":"2023-12-16T21:42:41.367605+0000","last_active":"2023-12-16T21:42:42.329203+0000","last_peered":"2023-12-16T21:42:42.329203+0000","last_clean":"2023-12-16T21:42:42.329203+0000","last_became_active":"2023-12-16T21:42:41.366781+0000","last_became_peered":"2023-12-16T21:42:41.366781+0000","last_unstale":"2023-12-16T21:42:42.329203+0000","last_undegraded":"2023-12-16T21:42:42.329203+0000","last_fullsized":"2023-12-16T21:42:42.329203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475185+0000","last_change":"2023-12-16T21:42:40.462161+0000","last_active":"2023-12-16T21:42:42.475185+0000","last_peered":"2023-12-16T21:42:42.475185+0000","last_clean":"2023-12-16T21:42:42.475185+0000","last_became_active":"2023-12-16T21:42:40.461535+0000","last_became_peered":"2023-12-16T21:42:40.461535+0000","last_unstale":"2023-12-16T21:42:42.475185+0000","last_undegraded":"2023-12-16T21:42:42.475185+0000","last_fullsized":"2023-12-16T21:42:42.475185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476429+0000","last_change":"2023-12-16T21:42:41.366419+0000","last_active":"2023-12-16T21:42:42.476429+0000","last_peered":"2023-12-16T21:42:42.476429+0000","last_clean":"2023-12-16T21:42:42.476429+0000","last_became_active":"2023-12-16T21:42:41.365644+0000","last_became_peered":"2023-12-16T21:42:41.365644+0000","last_unstale":"2023-12-16T21:42:42.476429+0000","last_undegraded":"2023-12-16T21:42:42.476429+0000","last_fullsized":"2023-12-16T21:42:42.476429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476308+0000","last_change":"2023-12-16T21:42:40.461870+0000","last_active":"2023-12-16T21:42:42.476308+0000","last_peered":"2023-12-16T21:42:42.476308+0000","last_clean":"2023-12-16T21:42:42.476308+0000","last_became_active":"2023-12-16T21:42:40.461189+0000","last_became_peered":"2023-12-16T21:42:40.461189+0000","last_unstale":"2023-12-16T21:42:42.476308+0000","last_undegraded":"2023-12-16T21:42:42.476308+0000","last_fullsized":"2023-12-16T21:42:42.476308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471926+0000","last_change":"2023-12-16T21:42:41.385522+0000","last_active":"2023-12-16T21:42:42.471926+0000","last_peered":"2023-12-16T21:42:42.471926+0000","last_clean":"2023-12-16T21:42:42.471926+0000","last_became_active":"2023-12-16T21:42:41.384952+0000","last_became_peered":"2023-12-16T21:42:41.384952+0000","last_unstale":"2023-12-16T21:42:42.471926+0000","last_undegraded":"2023-12-16T21:42:42.471926+0000","last_fullsized":"2023-12-16T21:42:42.471926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330102+0000","last_change":"2023-12-16T21:42:40.453133+0000","last_active":"2023-12-16T21:42:42.330102+0000","last_peered":"2023-12-16T21:42:42.330102+0000","last_clean":"2023-12-16T21:42:42.330102+0000","last_became_active":"2023-12-16T21:42:40.452814+0000","last_became_peered":"2023-12-16T21:42:40.452814+0000","last_unstale":"2023-12-16T21:42:42.330102+0000","last_undegraded":"2023-12-16T21:42:42.330102+0000","last_fullsized":"2023-12-16T21:42:42.330102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335567+0000","last_change":"2023-12-16T21:42:41.359165+0000","last_active":"2023-12-16T21:42:42.335567+0000","last_peered":"2023-12-16T21:42:42.335567+0000","last_clean":"2023-12-16T21:42:42.335567+0000","last_became_active":"2023-12-16T21:42:41.356738+0000","last_became_peered":"2023-12-16T21:42:41.356738+0000","last_unstale":"2023-12-16T21:42:42.335567+0000","last_undegraded":"2023-12-16T21:42:42.335567+0000","last_fullsized":"2023-12-16T21:42:42.335567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475530+0000","last_change":"2023-12-16T21:42:40.428771+0000","last_active":"2023-12-16T21:42:42.475530+0000","last_peered":"2023-12-16T21:42:42.475530+0000","last_clean":"2023-12-16T21:42:42.475530+0000","last_became_active":"2023-12-16T21:42:40.427921+0000","last_became_peered":"2023-12-16T21:42:40.427921+0000","last_unstale":"2023-12-16T21:42:42.475530+0000","last_undegraded":"2023-12-16T21:42:42.475530+0000","last_fullsized":"2023-12-16T21:42:42.475530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334711+0000","last_change":"2023-12-16T21:42:41.373637+0000","last_active":"2023-12-16T21:42:42.334711+0000","last_peered":"2023-12-16T21:42:42.334711+0000","last_clean":"2023-12-16T21:42:42.334711+0000","last_became_active":"2023-12-16T21:42:41.373003+0000","last_became_peered":"2023-12-16T21:42:41.373003+0000","last_unstale":"2023-12-16T21:42:42.334711+0000","last_undegraded":"2023-12-16T21:42:42.334711+0000","last_fullsized":"2023-12-16T21:42:42.334711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:48:55.034570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472336+0000","last_change":"2023-12-16T21:42:40.424774+0000","last_active":"2023-12-16T21:42:42.472336+0000","last_peered":"2023-12-16T21:42:42.472336+0000","last_clean":"2023-12-16T21:42:42.472336+0000","last_became_active":"2023-12-16T21:42:40.424333+0000","last_became_peered":"2023-12-16T21:42:40.424333+0000","last_unstale":"2023-12-16T21:42:42.472336+0000","last_undegraded":"2023-12-16T21:42:42.472336+0000","last_fullsized":"2023-12-16T21:42:42.472336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472450+0000","last_change":"2023-12-16T21:42:41.377224+0000","last_active":"2023-12-16T21:42:42.472450+0000","last_peered":"2023-12-16T21:42:42.472450+0000","last_clean":"2023-12-16T21:42:42.472450+0000","last_became_active":"2023-12-16T21:42:41.376574+0000","last_became_peered":"2023-12-16T21:42:41.376574+0000","last_unstale":"2023-12-16T21:42:42.472450+0000","last_undegraded":"2023-12-16T21:42:42.472450+0000","last_fullsized":"2023-12-16T21:42:42.472450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475529+0000","last_change":"2023-12-16T21:42:40.453599+0000","last_active":"2023-12-16T21:42:42.475529+0000","last_peered":"2023-12-16T21:42:42.475529+0000","last_clean":"2023-12-16T21:42:42.475529+0000","last_became_active":"2023-12-16T21:42:40.452662+0000","last_became_peered":"2023-12-16T21:42:40.452662+0000","last_unstale":"2023-12-16T21:42:42.475529+0000","last_undegraded":"2023-12-16T21:42:42.475529+0000","last_fullsized":"2023-12-16T21:42:42.475529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:02.018716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333919+0000","last_change":"2023-12-16T21:42:41.365750+0000","last_active":"2023-12-16T21:42:42.333919+0000","last_peered":"2023-12-16T21:42:42.333919+0000","last_clean":"2023-12-16T21:42:42.333919+0000","last_became_active":"2023-12-16T21:42:41.365097+0000","last_became_peered":"2023-12-16T21:42:41.365097+0000","last_unstale":"2023-12-16T21:42:42.333919+0000","last_undegraded":"2023-12-16T21:42:42.333919+0000","last_fullsized":"2023-12-16T21:42:42.333919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474655+0000","last_change":"2023-12-16T21:42:40.437256+0000","last_active":"2023-12-16T21:42:42.474655+0000","last_peered":"2023-12-16T21:42:42.474655+0000","last_clean":"2023-12-16T21:42:42.474655+0000","last_became_active":"2023-12-16T21:42:40.436394+0000","last_became_peered":"2023-12-16T21:42:40.436394+0000","last_unstale":"2023-12-16T21:42:42.474655+0000","last_undegraded":"2023-12-16T21:42:42.474655+0000","last_fullsized":"2023-12-16T21:42:42.474655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473943+0000","last_change":"2023-12-16T21:42:41.373882+0000","last_active":"2023-12-16T21:42:42.473943+0000","last_peered":"2023-12-16T21:42:42.473943+0000","last_clean":"2023-12-16T21:42:42.473943+0000","last_became_active":"2023-12-16T21:42:41.373250+0000","last_became_peered":"2023-12-16T21:42:41.373250+0000","last_unstale":"2023-12-16T21:42:42.473943+0000","last_undegraded":"2023-12-16T21:42:42.473943+0000","last_fullsized":"2023-12-16T21:42:42.473943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471379+0000","last_change":"2023-12-16T21:42:40.428292+0000","last_active":"2023-12-16T21:42:42.471379+0000","last_peered":"2023-12-16T21:42:42.471379+0000","last_clean":"2023-12-16T21:42:42.471379+0000","last_became_active":"2023-12-16T21:42:40.427611+0000","last_became_peered":"2023-12-16T21:42:40.427611+0000","last_unstale":"2023-12-16T21:42:42.471379+0000","last_undegraded":"2023-12-16T21:42:42.471379+0000","last_fullsized":"2023-12-16T21:42:42.471379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332822+0000","last_change":"2023-12-16T21:42:41.364395+0000","last_active":"2023-12-16T21:42:42.332822+0000","last_peered":"2023-12-16T21:42:42.332822+0000","last_clean":"2023-12-16T21:42:42.332822+0000","last_became_active":"2023-12-16T21:42:41.363596+0000","last_became_peered":"2023-12-16T21:42:41.363596+0000","last_unstale":"2023-12-16T21:42:42.332822+0000","last_undegraded":"2023-12-16T21:42:42.332822+0000","last_fullsized":"2023-12-16T21:42:42.332822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335857+0000","last_change":"2023-12-16T21:42:40.430896+0000","last_active":"2023-12-16T21:42:42.335857+0000","last_peered":"2023-12-16T21:42:42.335857+0000","last_clean":"2023-12-16T21:42:42.335857+0000","last_became_active":"2023-12-16T21:42:40.425814+0000","last_became_peered":"2023-12-16T21:42:40.425814+0000","last_unstale":"2023-12-16T21:42:42.335857+0000","last_undegraded":"2023-12-16T21:42:42.335857+0000","last_fullsized":"2023-12-16T21:42:42.335857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:39:34.410203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334498+0000","last_change":"2023-12-16T21:42:41.364963+0000","last_active":"2023-12-16T21:42:42.334498+0000","last_peered":"2023-12-16T21:42:42.334498+0000","last_clean":"2023-12-16T21:42:42.334498+0000","last_became_active":"2023-12-16T21:42:41.364207+0000","last_became_peered":"2023-12-16T21:42:41.364207+0000","last_unstale":"2023-12-16T21:42:42.334498+0000","last_undegraded":"2023-12-16T21:42:42.334498+0000","last_fullsized":"2023-12-16T21:42:42.334498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:11:32.433518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471333+0000","last_change":"2023-12-16T21:42:40.424410+0000","last_active":"2023-12-16T21:42:42.471333+0000","last_peered":"2023-12-16T21:42:42.471333+0000","last_clean":"2023-12-16T21:42:42.471333+0000","last_became_active":"2023-12-16T21:42:40.423891+0000","last_became_peered":"2023-12-16T21:42:40.423891+0000","last_unstale":"2023-12-16T21:42:42.471333+0000","last_undegraded":"2023-12-16T21:42:42.471333+0000","last_fullsized":"2023-12-16T21:42:42.471333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334300+0000","last_change":"2023-12-16T21:42:41.366442+0000","last_active":"2023-12-16T21:42:42.334300+0000","last_peered":"2023-12-16T21:42:42.334300+0000","last_clean":"2023-12-16T21:42:42.334300+0000","last_became_active":"2023-12-16T21:42:41.363759+0000","last_became_peered":"2023-12-16T21:42:41.363759+0000","last_unstale":"2023-12-16T21:42:42.334300+0000","last_undegraded":"2023-12-16T21:42:42.334300+0000","last_fullsized":"2023-12-16T21:42:42.334300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:44:20.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330332+0000","last_change":"2023-12-16T21:42:40.429084+0000","last_active":"2023-12-16T21:42:42.330332+0000","last_peered":"2023-12-16T21:42:42.330332+0000","last_clean":"2023-12-16T21:42:42.330332+0000","last_became_active":"2023-12-16T21:42:40.428576+0000","last_became_peered":"2023-12-16T21:42:40.428576+0000","last_unstale":"2023-12-16T21:42:42.330332+0000","last_undegraded":"2023-12-16T21:42:42.330332+0000","last_fullsized":"2023-12-16T21:42:42.330332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475041+0000","last_change":"2023-12-16T21:42:41.385053+0000","last_active":"2023-12-16T21:42:42.475041+0000","last_peered":"2023-12-16T21:42:42.475041+0000","last_clean":"2023-12-16T21:42:42.475041+0000","last_became_active":"2023-12-16T21:42:41.384196+0000","last_became_peered":"2023-12-16T21:42:41.384196+0000","last_unstale":"2023-12-16T21:42:42.475041+0000","last_undegraded":"2023-12-16T21:42:42.475041+0000","last_fullsized":"2023-12-16T21:42:42.475041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471297+0000","last_change":"2023-12-16T21:42:40.461203+0000","last_active":"2023-12-16T21:42:42.471297+0000","last_peered":"2023-12-16T21:42:42.471297+0000","last_clean":"2023-12-16T21:42:42.471297+0000","last_became_active":"2023-12-16T21:42:40.460912+0000","last_became_peered":"2023-12-16T21:42:40.460912+0000","last_unstale":"2023-12-16T21:42:42.471297+0000","last_undegraded":"2023-12-16T21:42:42.471297+0000","last_fullsized":"2023-12-16T21:42:42.471297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:51.118596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335069+0000","last_change":"2023-12-16T21:42:41.369186+0000","last_active":"2023-12-16T21:42:42.335069+0000","last_peered":"2023-12-16T21:42:42.335069+0000","last_clean":"2023-12-16T21:42:42.335069+0000","last_became_active":"2023-12-16T21:42:41.368337+0000","last_became_peered":"2023-12-16T21:42:41.368337+0000","last_unstale":"2023-12-16T21:42:42.335069+0000","last_undegraded":"2023-12-16T21:42:42.335069+0000","last_fullsized":"2023-12-16T21:42:42.335069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:57:56.730962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-16T21:42:41.331102+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:42:41.331102+0000","last_peered":"2023-12-16T21:42:41.331102+0000","last_clean":"2023-12-16T21:42:41.331102+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:42:41.331102+0000","last_undegraded":"2023-12-16T21:42:41.331102+0000","last_fullsized":"2023-12-16T21:42:41.331102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474341+0000","last_change":"2023-12-16T21:42:40.457329+0000","last_active":"2023-12-16T21:42:42.474341+0000","last_peered":"2023-12-16T21:42:42.474341+0000","last_clean":"2023-12-16T21:42:42.474341+0000","last_became_active":"2023-12-16T21:42:40.453943+0000","last_became_peered":"2023-12-16T21:42:40.453943+0000","last_unstale":"2023-12-16T21:42:42.474341+0000","last_undegraded":"2023-12-16T21:42:42.474341+0000","last_fullsized":"2023-12-16T21:42:42.474341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472035+0000","last_change":"2023-12-16T21:42:41.367983+0000","last_active":"2023-12-16T21:42:42.472035+0000","last_peered":"2023-12-16T21:42:42.472035+0000","last_clean":"2023-12-16T21:42:42.472035+0000","last_became_active":"2023-12-16T21:42:41.367201+0000","last_became_peered":"2023-12-16T21:42:41.367201+0000","last_unstale":"2023-12-16T21:42:42.472035+0000","last_undegraded":"2023-12-16T21:42:42.472035+0000","last_fullsized":"2023-12-16T21:42:42.472035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332137+0000","last_change":"2023-12-16T21:42:40.447913+0000","last_active":"2023-12-16T21:42:42.332137+0000","last_peered":"2023-12-16T21:42:42.332137+0000","last_clean":"2023-12-16T21:42:42.332137+0000","last_became_active":"2023-12-16T21:42:40.445110+0000","last_became_peered":"2023-12-16T21:42:40.445110+0000","last_unstale":"2023-12-16T21:42:42.332137+0000","last_undegraded":"2023-12-16T21:42:42.332137+0000","last_fullsized":"2023-12-16T21:42:42.332137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471511+0000","last_change":"2023-12-16T21:42:41.378992+0000","last_active":"2023-12-16T21:42:42.471511+0000","last_peered":"2023-12-16T21:42:42.471511+0000","last_clean":"2023-12-16T21:42:42.471511+0000","last_became_active":"2023-12-16T21:42:41.378192+0000","last_became_peered":"2023-12-16T21:42:41.378192+0000","last_unstale":"2023-12-16T21:42:42.471511+0000","last_undegraded":"2023-12-16T21:42:42.471511+0000","last_fullsized":"2023-12-16T21:42:42.471511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:42.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472434+0000","last_change":"2023-12-16T21:42:40.424908+0000","last_active":"2023-12-16T21:42:42.472434+0000","last_peered":"2023-12-16T21:42:42.472434+0000","last_clean":"2023-12-16T21:42:42.472434+0000","last_became_active":"2023-12-16T21:42:40.424358+0000","last_became_peered":"2023-12-16T21:42:40.424358+0000","last_unstale":"2023-12-16T21:42:42.472434+0000","last_undegraded":"2023-12-16T21:42:42.472434+0000","last_fullsized":"2023-12-16T21:42:42.472434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329469+0000","last_change":"2023-12-16T21:42:41.352792+0000","last_active":"2023-12-16T21:42:42.329469+0000","last_peered":"2023-12-16T21:42:42.329469+0000","last_clean":"2023-12-16T21:42:42.329469+0000","last_became_active":"2023-12-16T21:42:41.352078+0000","last_became_peered":"2023-12-16T21:42:41.352078+0000","last_unstale":"2023-12-16T21:42:42.329469+0000","last_undegraded":"2023-12-16T21:42:42.329469+0000","last_fullsized":"2023-12-16T21:42:42.329469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475741+0000","last_change":"2023-12-16T21:42:40.457237+0000","last_active":"2023-12-16T21:42:42.475741+0000","last_peered":"2023-12-16T21:42:42.475741+0000","last_clean":"2023-12-16T21:42:42.475741+0000","last_became_active":"2023-12-16T21:42:40.452459+0000","last_became_peered":"2023-12-16T21:42:40.452459+0000","last_unstale":"2023-12-16T21:42:42.475741+0000","last_undegraded":"2023-12-16T21:42:42.475741+0000","last_fullsized":"2023-12-16T21:42:42.475741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:57.940854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334555+0000","last_change":"2023-12-16T21:42:41.352263+0000","last_active":"2023-12-16T21:42:42.334555+0000","last_peered":"2023-12-16T21:42:42.334555+0000","last_clean":"2023-12-16T21:42:42.334555+0000","last_became_active":"2023-12-16T21:42:41.347293+0000","last_became_peered":"2023-12-16T21:42:41.347293+0000","last_unstale":"2023-12-16T21:42:42.334555+0000","last_undegraded":"2023-12-16T21:42:42.334555+0000","last_fullsized":"2023-12-16T21:42:42.334555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:52.699335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329677+0000","last_change":"2023-12-16T21:42:40.423235+0000","last_active":"2023-12-16T21:42:42.329677+0000","last_peered":"2023-12-16T21:42:42.329677+0000","last_clean":"2023-12-16T21:42:42.329677+0000","last_became_active":"2023-12-16T21:42:40.422630+0000","last_became_peered":"2023-12-16T21:42:40.422630+0000","last_unstale":"2023-12-16T21:42:42.329677+0000","last_undegraded":"2023-12-16T21:42:42.329677+0000","last_fullsized":"2023-12-16T21:42:42.329677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475779+0000","last_change":"2023-12-16T21:42:41.372994+0000","last_active":"2023-12-16T21:42:42.475779+0000","last_peered":"2023-12-16T21:42:42.475779+0000","last_clean":"2023-12-16T21:42:42.475779+0000","last_became_active":"2023-12-16T21:42:41.372135+0000","last_became_peered":"2023-12-16T21:42:41.372135+0000","last_unstale":"2023-12-16T21:42:42.475779+0000","last_undegraded":"2023-12-16T21:42:42.475779+0000","last_fullsized":"2023-12-16T21:42:42.475779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335404+0000","last_change":"2023-12-16T21:42:40.445536+0000","last_active":"2023-12-16T21:42:42.335404+0000","last_peered":"2023-12-16T21:42:42.335404+0000","last_clean":"2023-12-16T21:42:42.335404+0000","last_became_active":"2023-12-16T21:42:40.444652+0000","last_became_peered":"2023-12-16T21:42:40.444652+0000","last_unstale":"2023-12-16T21:42:42.335404+0000","last_undegraded":"2023-12-16T21:42:42.335404+0000","last_fullsized":"2023-12-16T21:42:42.335404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:38.836527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329548+0000","last_change":"2023-12-16T21:42:41.352754+0000","last_active":"2023-12-16T21:42:42.329548+0000","last_peered":"2023-12-16T21:42:42.329548+0000","last_clean":"2023-12-16T21:42:42.329548+0000","last_became_active":"2023-12-16T21:42:41.351903+0000","last_became_peered":"2023-12-16T21:42:41.351903+0000","last_unstale":"2023-12-16T21:42:42.329548+0000","last_undegraded":"2023-12-16T21:42:42.329548+0000","last_fullsized":"2023-12-16T21:42:42.329548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.687428+0000","last_change":"2023-12-16T21:42:40.425139+0000","last_active":"2023-12-16T21:42:42.687428+0000","last_peered":"2023-12-16T21:42:42.687428+0000","last_clean":"2023-12-16T21:42:42.687428+0000","last_became_active":"2023-12-16T21:42:40.423820+0000","last_became_peered":"2023-12-16T21:42:40.423820+0000","last_unstale":"2023-12-16T21:42:42.687428+0000","last_undegraded":"2023-12-16T21:42:42.687428+0000","last_fullsized":"2023-12-16T21:42:42.687428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:37.970083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472944+0000","last_change":"2023-12-16T21:42:41.365965+0000","last_active":"2023-12-16T21:42:42.472944+0000","last_peered":"2023-12-16T21:42:42.472944+0000","last_clean":"2023-12-16T21:42:42.472944+0000","last_became_active":"2023-12-16T21:42:41.365191+0000","last_became_peered":"2023-12-16T21:42:41.365191+0000","last_unstale":"2023-12-16T21:42:42.472944+0000","last_undegraded":"2023-12-16T21:42:42.472944+0000","last_fullsized":"2023-12-16T21:42:42.472944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476214+0000","last_change":"2023-12-16T21:42:40.447186+0000","last_active":"2023-12-16T21:42:42.476214+0000","last_peered":"2023-12-16T21:42:42.476214+0000","last_clean":"2023-12-16T21:42:42.476214+0000","last_became_active":"2023-12-16T21:42:40.446374+0000","last_became_peered":"2023-12-16T21:42:40.446374+0000","last_unstale":"2023-12-16T21:42:42.476214+0000","last_undegraded":"2023-12-16T21:42:42.476214+0000","last_fullsized":"2023-12-16T21:42:42.476214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471782+0000","last_change":"2023-12-16T21:42:41.364750+0000","last_active":"2023-12-16T21:42:42.471782+0000","last_peered":"2023-12-16T21:42:42.471782+0000","last_clean":"2023-12-16T21:42:42.471782+0000","last_became_active":"2023-12-16T21:42:41.363976+0000","last_became_peered":"2023-12-16T21:42:41.363976+0000","last_unstale":"2023-12-16T21:42:42.471782+0000","last_undegraded":"2023-12-16T21:42:42.471782+0000","last_fullsized":"2023-12-16T21:42:42.471782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332140+0000","last_change":"2023-12-16T21:42:40.444481+0000","last_active":"2023-12-16T21:42:42.332140+0000","last_peered":"2023-12-16T21:42:42.332140+0000","last_clean":"2023-12-16T21:42:42.332140+0000","last_became_active":"2023-12-16T21:42:40.443665+0000","last_became_peered":"2023-12-16T21:42:40.443665+0000","last_unstale":"2023-12-16T21:42:42.332140+0000","last_undegraded":"2023-12-16T21:42:42.332140+0000","last_fullsized":"2023-12-16T21:42:42.332140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334181+0000","last_change":"2023-12-16T21:42:41.360039+0000","last_active":"2023-12-16T21:42:42.334181+0000","last_peered":"2023-12-16T21:42:42.334181+0000","last_clean":"2023-12-16T21:42:42.334181+0000","last_became_active":"2023-12-16T21:42:41.359132+0000","last_became_peered":"2023-12-16T21:42:41.359132+0000","last_unstale":"2023-12-16T21:42:42.334181+0000","last_undegraded":"2023-12-16T21:42:42.334181+0000","last_fullsized":"2023-12-16T21:42:42.334181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:52.703815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329301+0000","last_change":"2023-12-16T21:42:40.438910+0000","last_active":"2023-12-16T21:42:42.329301+0000","last_peered":"2023-12-16T21:42:42.329301+0000","last_clean":"2023-12-16T21:42:42.329301+0000","last_became_active":"2023-12-16T21:42:40.438462+0000","last_became_peered":"2023-12-16T21:42:40.438462+0000","last_unstale":"2023-12-16T21:42:42.329301+0000","last_undegraded":"2023-12-16T21:42:42.329301+0000","last_fullsized":"2023-12-16T21:42:42.329301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472184+0000","last_change":"2023-12-16T21:42:41.364881+0000","last_active":"2023-12-16T21:42:42.472184+0000","last_peered":"2023-12-16T21:42:42.472184+0000","last_clean":"2023-12-16T21:42:42.472184+0000","last_became_active":"2023-12-16T21:42:41.363545+0000","last_became_peered":"2023-12-16T21:42:41.363545+0000","last_unstale":"2023-12-16T21:42:42.472184+0000","last_undegraded":"2023-12-16T21:42:42.472184+0000","last_fullsized":"2023-12-16T21:42:42.472184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329671+0000","last_change":"2023-12-16T21:42:40.418011+0000","last_active":"2023-12-16T21:42:42.329671+0000","last_peered":"2023-12-16T21:42:42.329671+0000","last_clean":"2023-12-16T21:42:42.329671+0000","last_became_active":"2023-12-16T21:42:40.417587+0000","last_became_peered":"2023-12-16T21:42:40.417587+0000","last_unstale":"2023-12-16T21:42:42.329671+0000","last_undegraded":"2023-12-16T21:42:42.329671+0000","last_fullsized":"2023-12-16T21:42:42.329671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472662+0000","last_change":"2023-12-16T21:42:41.380236+0000","last_active":"2023-12-16T21:42:42.472662+0000","last_peered":"2023-12-16T21:42:42.472662+0000","last_clean":"2023-12-16T21:42:42.472662+0000","last_became_active":"2023-12-16T21:42:41.379681+0000","last_became_peered":"2023-12-16T21:42:41.379681+0000","last_unstale":"2023-12-16T21:42:42.472662+0000","last_undegraded":"2023-12-16T21:42:42.472662+0000","last_fullsized":"2023-12-16T21:42:42.472662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334158+0000","last_change":"2023-12-16T21:42:40.448985+0000","last_active":"2023-12-16T21:42:42.334158+0000","last_peered":"2023-12-16T21:42:42.334158+0000","last_clean":"2023-12-16T21:42:42.334158+0000","last_became_active":"2023-12-16T21:42:40.443742+0000","last_became_peered":"2023-12-16T21:42:40.443742+0000","last_unstale":"2023-12-16T21:42:42.334158+0000","last_undegraded":"2023-12-16T21:42:42.334158+0000","last_fullsized":"2023-12-16T21:42:42.334158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:53.161999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474831+0000","last_change":"2023-12-16T21:42:41.363951+0000","last_active":"2023-12-16T21:42:42.474831+0000","last_peered":"2023-12-16T21:42:42.474831+0000","last_clean":"2023-12-16T21:42:42.474831+0000","last_became_active":"2023-12-16T21:42:41.363161+0000","last_became_peered":"2023-12-16T21:42:41.363161+0000","last_unstale":"2023-12-16T21:42:42.474831+0000","last_undegraded":"2023-12-16T21:42:42.474831+0000","last_fullsized":"2023-12-16T21:42:42.474831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475088+0000","last_change":"2023-12-16T21:42:41.386385+0000","last_active":"2023-12-16T21:42:42.475088+0000","last_peered":"2023-12-16T21:42:42.475088+0000","last_clean":"2023-12-16T21:42:42.475088+0000","last_became_active":"2023-12-16T21:42:41.385339+0000","last_became_peered":"2023-12-16T21:42:41.385339+0000","last_unstale":"2023-12-16T21:42:42.475088+0000","last_undegraded":"2023-12-16T21:42:42.475088+0000","last_fullsized":"2023-12-16T21:42:42.475088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332799+0000","last_change":"2023-12-16T21:42:40.462751+0000","last_active":"2023-12-16T21:42:42.332799+0000","last_peered":"2023-12-16T21:42:42.332799+0000","last_clean":"2023-12-16T21:42:42.332799+0000","last_became_active":"2023-12-16T21:42:40.461936+0000","last_became_peered":"2023-12-16T21:42:40.461936+0000","last_unstale":"2023-12-16T21:42:42.332799+0000","last_undegraded":"2023-12-16T21:42:42.332799+0000","last_fullsized":"2023-12-16T21:42:42.332799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474683+0000","last_change":"2023-12-16T21:42:41.384715+0000","last_active":"2023-12-16T21:42:42.474683+0000","last_peered":"2023-12-16T21:42:42.474683+0000","last_clean":"2023-12-16T21:42:42.474683+0000","last_became_active":"2023-12-16T21:42:41.383596+0000","last_became_peered":"2023-12-16T21:42:41.383596+0000","last_unstale":"2023-12-16T21:42:42.474683+0000","last_undegraded":"2023-12-16T21:42:42.474683+0000","last_fullsized":"2023-12-16T21:42:42.474683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335044+0000","last_change":"2023-12-16T21:42:40.445218+0000","last_active":"2023-12-16T21:42:42.335044+0000","last_peered":"2023-12-16T21:42:42.335044+0000","last_clean":"2023-12-16T21:42:42.335044+0000","last_became_active":"2023-12-16T21:42:40.443892+0000","last_became_peered":"2023-12-16T21:42:40.443892+0000","last_unstale":"2023-12-16T21:42:42.335044+0000","last_undegraded":"2023-12-16T21:42:42.335044+0000","last_fullsized":"2023-12-16T21:42:42.335044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:20.422855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471754+0000","last_change":"2023-12-16T21:42:41.385409+0000","last_active":"2023-12-16T21:42:42.471754+0000","last_peered":"2023-12-16T21:42:42.471754+0000","last_clean":"2023-12-16T21:42:42.471754+0000","last_became_active":"2023-12-16T21:42:41.384925+0000","last_became_peered":"2023-12-16T21:42:41.384925+0000","last_unstale":"2023-12-16T21:42:42.471754+0000","last_undegraded":"2023-12-16T21:42:42.471754+0000","last_fullsized":"2023-12-16T21:42:42.471754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472993+0000","last_change":"2023-12-16T21:42:40.417361+0000","last_active":"2023-12-16T21:42:42.472993+0000","last_peered":"2023-12-16T21:42:42.472993+0000","last_clean":"2023-12-16T21:42:42.472993+0000","last_became_active":"2023-12-16T21:42:40.416759+0000","last_became_peered":"2023-12-16T21:42:40.416759+0000","last_unstale":"2023-12-16T21:42:42.472993+0000","last_undegraded":"2023-12-16T21:42:42.472993+0000","last_fullsized":"2023-12-16T21:42:42.472993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472885+0000","last_change":"2023-12-16T21:42:41.370637+0000","last_active":"2023-12-16T21:42:42.472885+0000","last_peered":"2023-12-16T21:42:42.472885+0000","last_clean":"2023-12-16T21:42:42.472885+0000","last_became_active":"2023-12-16T21:42:41.369929+0000","last_became_peered":"2023-12-16T21:42:41.369929+0000","last_unstale":"2023-12-16T21:42:42.472885+0000","last_undegraded":"2023-12-16T21:42:42.472885+0000","last_fullsized":"2023-12-16T21:42:42.472885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471919+0000","last_change":"2023-12-16T21:42:40.439054+0000","last_active":"2023-12-16T21:42:42.471919+0000","last_peered":"2023-12-16T21:42:42.471919+0000","last_clean":"2023-12-16T21:42:42.471919+0000","last_became_active":"2023-12-16T21:42:40.438547+0000","last_became_peered":"2023-12-16T21:42:40.438547+0000","last_unstale":"2023-12-16T21:42:42.471919+0000","last_undegraded":"2023-12-16T21:42:42.471919+0000","last_fullsized":"2023-12-16T21:42:42.471919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:42:47.078 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:42:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.227+0000 7f44054ac640 1 Processor -- start 2023-12-16T21:42:47.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f44054ac640 1 -- start start 2023-12-16T21:42:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f44054ac640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f440006b730 0x7f44001432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f44054ac640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4400143820 0x7f4400143cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f44054ac640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44000ac8f0 con 0x7f44000736b0 2023-12-16T21:42:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f44054ac640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44000aceb0 con 0x7f440006b730 2023-12-16T21:42:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f44054ac640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44000ae040 con 0x7f4400143820 2023-12-16T21:42:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f440006b730 0x7f44001432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f440006b730 0x7f44001432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45236/0 (socket says 172.21.15.46:45236) 2023-12-16T21:42:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 -- 172.21.15.46:0/1725865060 learned_addr learned my addr 172.21.15.46:0/1725865060 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43ff7fe640 1 --2- 172.21.15.46:0/1725865060 >> v2:172.21.15.161:3300/0 conn(0x7f4400143820 0x7f4400143cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 -- 172.21.15.46:0/1725865060 >> v2:172.21.15.161:3300/0 conn(0x7f4400143820 msgr2=0x7f4400143cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 --2- 172.21.15.46:0/1725865060 >> v2:172.21.15.161:3300/0 conn(0x7f4400143820 0x7f4400143cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 -- 172.21.15.46:0/1725865060 >> v1:172.21.15.161:6789/0 conn(0x7f44000736b0 legacy=0x7f44000ad930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 -- 172.21.15.46:0/1725865060 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400073a90 con 0x7f440006b730 2023-12-16T21:42:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fe7fc640 1 --2- 172.21.15.46:0/1725865060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440006b730 0x7f44001432e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009870 tx=0x7f43f401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3653dd80a4e97 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fdffb640 1 -- 172.21.15.46:0/1725865060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40ac410 con 0x7f440006b730 2023-12-16T21:42:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fdffb640 1 -- 172.21.15.46:0/1725865060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f40ac570 con 0x7f440006b730 2023-12-16T21:42:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.231+0000 7f43fdffb640 1 -- 172.21.15.46:0/1725865060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40ac900 con 0x7f440006b730 2023-12-16T21:42:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- 172.21.15.46:0/1725865060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440006b730 msgr2=0x7f44001432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 --2- 172.21.15.46:0/1725865060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440006b730 0x7f44001432e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009870 tx=0x7f43f401c3b0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- 172.21.15.46:0/1725865060 shutdown_connections 2023-12-16T21:42:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 --2- 172.21.15.46:0/1725865060 >> v2:172.21.15.161:3300/0 conn(0x7f4400143820 0x7f4400143cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 --2- 172.21.15.46:0/1725865060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f440006b730 0x7f44001432e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- 172.21.15.46:0/1725865060 >> 172.21.15.46:0/1725865060 conn(0x7f440006a580 msgr2=0x7f440006a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- 172.21.15.46:0/1725865060 shutdown_connections 2023-12-16T21:42:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- 172.21.15.46:0/1725865060 wait complete. 2023-12-16T21:42:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 Processor -- start 2023-12-16T21:42:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- start start 2023-12-16T21:42:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f440006b730 0x7f440013da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44000736b0 0x7f44001e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400143820 0x7f4400135430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44000b29a0 con 0x7f44000736b0 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44001e1cf0 con 0x7f4400143820 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f44054ac640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440006ff10 con 0x7f440006b730 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44000736b0 0x7f44001e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44000736b0 0x7f44001e21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45252/0 (socket says 172.21.15.46:45252) 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 -- 172.21.15.46:0/525182640 learned_addr learned my addr 172.21.15.46:0/525182640 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43feffd640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f440006b730 0x7f440013da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43ff7fe640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400143820 0x7f4400135430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 -- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f440006b730 msgr2=0x7f440013da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f440006b730 0x7f440013da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 -- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400143820 msgr2=0x7f4400135430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400143820 0x7f4400135430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 -- 172.21.15.46:0/525182640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f40094d0 con 0x7f44000736b0 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43ff7fe640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400143820 0x7f4400135430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43fe7fc640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44000736b0 0x7f44001e21d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f43f40af040 tx=0x7f43f4019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43e77fe640 1 -- 172.21.15.46:0/525182640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40ac410 con 0x7f44000736b0 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43e77fe640 1 -- 172.21.15.46:0/525182640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f40b65f0 con 0x7f44000736b0 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.235+0000 7f43e77fe640 1 -- 172.21.15.46:0/525182640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40b69a0 con 0x7f44000736b0 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44000b26c0 con 0x7f44000736b0 2023-12-16T21:42:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44000b23d0 con 0x7f44000736b0 2023-12-16T21:42:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44001e1b50 con 0x7f44000736b0 2023-12-16T21:42:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f43e77fe640 1 -- 172.21.15.46:0/525182640 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f43f40b6b00 con 0x7f44000736b0 2023-12-16T21:42:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f43e77fe640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f43ec066470 0x7f43ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f43e77fe640 1 -- 172.21.15.46:0/525182640 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43f411e8d0 con 0x7f44000736b0 2023-12-16T21:42:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f43feffd640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f43ec066470 0x7f43ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.239+0000 7f43feffd640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f43ec066470 0x7f43ec068930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4400072900 tx=0x7f43f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.243+0000 7f43e77fe640 1 -- 172.21.15.46:0/525182640 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f44001e1b50 con 0x7f44000736b0 2023-12-16T21:42:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.391+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f440013f910 con 0x7f43ec066470 2023-12-16T21:42:47.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.411+0000 7f43e77fe640 1 -- 172.21.15.46:0/525182640 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264573 (secure 0 0 0) 0x7f440013f910 con 0x7f43ec066470 2023-12-16T21:42:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:47.422 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:42:47.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f43ec066470 msgr2=0x7f43ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f43ec066470 0x7f43ec068930 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4400072900 tx=0x7f43f8009040 comp rx=0 tx=0).stop 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44000736b0 msgr2=0x7f44001e21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44000736b0 0x7f44001e21d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f43f40af040 tx=0x7f43f4019150 comp rx=0 tx=0).stop 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 shutdown_connections 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f43ec066470 0x7f43ec068930 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4400143820 0x7f4400135430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44000736b0 0x7f44001e21d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 --2- 172.21.15.46:0/525182640 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f440006b730 0x7f440013da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 >> 172.21.15.46:0/525182640 conn(0x7f440006a580 msgr2=0x7f440013f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 shutdown_connections 2023-12-16T21:42:47.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.415+0000 7f44054ac640 1 -- 172.21.15.46:0/525182640 wait complete. 2023-12-16T21:42:47.437 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-16T21:42:46.352226+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235764,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738956,"statfs":{"total":773094113280,"available":772852690944,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001862"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333636+0000","last_change":"2023-12-16T21:42:41.368574+0000","last_active":"2023-12-16T21:42:42.333636+0000","last_peered":"2023-12-16T21:42:42.333636+0000","last_clean":"2023-12-16T21:42:42.333636+0000","last_became_active":"2023-12-16T21:42:41.367555+0000","last_became_peered":"2023-12-16T21:42:41.367555+0000","last_unstale":"2023-12-16T21:42:42.333636+0000","last_undegraded":"2023-12-16T21:42:42.333636+0000","last_fullsized":"2023-12-16T21:42:42.333636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329805+0000","last_change":"2023-12-16T21:42:41.344487+0000","last_active":"2023-12-16T21:42:42.329805+0000","last_peered":"2023-12-16T21:42:42.329805+0000","last_clean":"2023-12-16T21:42:42.329805+0000","last_became_active":"2023-12-16T21:42:41.343798+0000","last_became_peered":"2023-12-16T21:42:41.343798+0000","last_unstale":"2023-12-16T21:42:42.329805+0000","last_undegraded":"2023-12-16T21:42:42.329805+0000","last_fullsized":"2023-12-16T21:42:42.329805+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471802+0000","last_change":"2023-12-16T21:42:41.364221+0000","last_active":"2023-12-16T21:42:42.471802+0000","last_peered":"2023-12-16T21:42:42.471802+0000","last_clean":"2023-12-16T21:42:42.471802+0000","last_became_active":"2023-12-16T21:42:41.362506+0000","last_became_peered":"2023-12-16T21:42:41.362506+0000","last_unstale":"2023-12-16T21:42:42.471802+0000","last_undegraded":"2023-12-16T21:42:42.471802+0000","last_fullsized":"2023-12-16T21:42:42.471802+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474631+0000","last_change":"2023-12-16T21:42:41.384692+0000","last_active":"2023-12-16T21:42:42.474631+0000","last_peered":"2023-12-16T21:42:42.474631+0000","last_clean":"2023-12-16T21:42:42.474631+0000","last_became_active":"2023-12-16T21:42:41.384156+0000","last_became_peered":"2023-12-16T21:42:41.384156+0000","last_unstale":"2023-12-16T21:42:42.474631+0000","last_undegraded":"2023-12-16T21:42:42.474631+0000","last_fullsized":"2023-12-16T21:42:42.474631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333576+0000","last_change":"2023-12-16T21:42:41.355491+0000","last_active":"2023-12-16T21:42:42.333576+0000","last_peered":"2023-12-16T21:42:42.333576+0000","last_clean":"2023-12-16T21:42:42.333576+0000","last_became_active":"2023-12-16T21:42:41.354519+0000","last_became_peered":"2023-12-16T21:42:41.354519+0000","last_unstale":"2023-12-16T21:42:42.333576+0000","last_undegraded":"2023-12-16T21:42:42.333576+0000","last_fullsized":"2023-12-16T21:42:42.333576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474628+0000","last_change":"2023-12-16T21:42:41.367454+0000","last_active":"2023-12-16T21:42:42.474628+0000","last_peered":"2023-12-16T21:42:42.474628+0000","last_clean":"2023-12-16T21:42:42.474628+0000","last_became_active":"2023-12-16T21:42:41.366663+0000","last_became_peered":"2023-12-16T21:42:41.366663+0000","last_unstale":"2023-12-16T21:42:42.474628+0000","last_undegraded":"2023-12-16T21:42:42.474628+0000","last_fullsized":"2023-12-16T21:42:42.474628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332459+0000","last_change":"2023-12-16T21:42:41.375370+0000","last_active":"2023-12-16T21:42:42.332459+0000","last_peered":"2023-12-16T21:42:42.332459+0000","last_clean":"2023-12-16T21:42:42.332459+0000","last_became_active":"2023-12-16T21:42:41.374733+0000","last_became_peered":"2023-12-16T21:42:41.374733+0000","last_unstale":"2023-12-16T21:42:42.332459+0000","last_undegraded":"2023-12-16T21:42:42.332459+0000","last_fullsized":"2023-12-16T21:42:42.332459+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471989+0000","last_change":"2023-12-16T21:42:41.385909+0000","last_active":"2023-12-16T21:42:42.471989+0000","last_peered":"2023-12-16T21:42:42.471989+0000","last_clean":"2023-12-16T21:42:42.471989+0000","last_became_active":"2023-12-16T21:42:41.385386+0000","last_became_peered":"2023-12-16T21:42:41.385386+0000","last_unstale":"2023-12-16T21:42:42.471989+0000","last_undegraded":"2023-12-16T21:42:42.471989+0000","last_fullsized":"2023-12-16T21:42:42.471989+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333967+0000","last_change":"2023-12-16T21:42:41.374260+0000","last_active":"2023-12-16T21:42:42.333967+0000","last_peered":"2023-12-16T21:42:42.333967+0000","last_clean":"2023-12-16T21:42:42.333967+0000","last_became_active":"2023-12-16T21:42:41.373101+0000","last_became_peered":"2023-12-16T21:42:41.373101+0000","last_unstale":"2023-12-16T21:42:42.333967+0000","last_undegraded":"2023-12-16T21:42:42.333967+0000","last_fullsized":"2023-12-16T21:42:42.333967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475842+0000","last_change":"2023-12-16T21:42:41.386312+0000","last_active":"2023-12-16T21:42:42.475842+0000","last_peered":"2023-12-16T21:42:42.475842+0000","last_clean":"2023-12-16T21:42:42.475842+0000","last_became_active":"2023-12-16T21:42:41.385651+0000","last_became_peered":"2023-12-16T21:42:41.385651+0000","last_unstale":"2023-12-16T21:42:42.475842+0000","last_undegraded":"2023-12-16T21:42:42.475842+0000","last_fullsized":"2023-12-16T21:42:42.475842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475260+0000","last_change":"2023-12-16T21:42:41.385382+0000","last_active":"2023-12-16T21:42:42.475260+0000","last_peered":"2023-12-16T21:42:42.475260+0000","last_clean":"2023-12-16T21:42:42.475260+0000","last_became_active":"2023-12-16T21:42:41.384671+0000","last_became_peered":"2023-12-16T21:42:41.384671+0000","last_unstale":"2023-12-16T21:42:42.475260+0000","last_undegraded":"2023-12-16T21:42:42.475260+0000","last_fullsized":"2023-12-16T21:42:42.475260+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335258+0000","last_change":"2023-12-16T21:42:41.373439+0000","last_active":"2023-12-16T21:42:42.335258+0000","last_peered":"2023-12-16T21:42:42.335258+0000","last_clean":"2023-12-16T21:42:42.335258+0000","last_became_active":"2023-12-16T21:42:41.372462+0000","last_became_peered":"2023-12-16T21:42:41.372462+0000","last_unstale":"2023-12-16T21:42:42.335258+0000","last_undegraded":"2023-12-16T21:42:42.335258+0000","last_fullsized":"2023-12-16T21:42:42.335258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471349+0000","last_change":"2023-12-16T21:42:41.366622+0000","last_active":"2023-12-16T21:42:42.471349+0000","last_peered":"2023-12-16T21:42:42.471349+0000","last_clean":"2023-12-16T21:42:42.471349+0000","last_became_active":"2023-12-16T21:42:41.365732+0000","last_became_peered":"2023-12-16T21:42:41.365732+0000","last_unstale":"2023-12-16T21:42:42.471349+0000","last_undegraded":"2023-12-16T21:42:42.471349+0000","last_fullsized":"2023-12-16T21:42:42.471349+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472312+0000","last_change":"2023-12-16T21:42:41.347792+0000","last_active":"2023-12-16T21:42:42.472312+0000","last_peered":"2023-12-16T21:42:42.472312+0000","last_clean":"2023-12-16T21:42:42.472312+0000","last_became_active":"2023-12-16T21:42:41.345588+0000","last_became_peered":"2023-12-16T21:42:41.345588+0000","last_unstale":"2023-12-16T21:42:42.472312+0000","last_undegraded":"2023-12-16T21:42:42.472312+0000","last_fullsized":"2023-12-16T21:42:42.472312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329364+0000","last_change":"2023-12-16T21:42:41.343823+0000","last_active":"2023-12-16T21:42:42.329364+0000","last_peered":"2023-12-16T21:42:42.329364+0000","last_clean":"2023-12-16T21:42:42.329364+0000","last_became_active":"2023-12-16T21:42:41.343022+0000","last_became_peered":"2023-12-16T21:42:41.343022+0000","last_unstale":"2023-12-16T21:42:42.329364+0000","last_undegraded":"2023-12-16T21:42:42.329364+0000","last_fullsized":"2023-12-16T21:42:42.329364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474121+0000","last_change":"2023-12-16T21:42:41.383653+0000","last_active":"2023-12-16T21:42:42.474121+0000","last_peered":"2023-12-16T21:42:42.474121+0000","last_clean":"2023-12-16T21:42:42.474121+0000","last_became_active":"2023-12-16T21:42:41.383223+0000","last_became_peered":"2023-12-16T21:42:41.383223+0000","last_unstale":"2023-12-16T21:42:42.474121+0000","last_undegraded":"2023-12-16T21:42:42.474121+0000","last_fullsized":"2023-12-16T21:42:42.474121+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476620+0000","last_change":"2023-12-16T21:42:41.366575+0000","last_active":"2023-12-16T21:42:42.476620+0000","last_peered":"2023-12-16T21:42:42.476620+0000","last_clean":"2023-12-16T21:42:42.476620+0000","last_became_active":"2023-12-16T21:42:41.365758+0000","last_became_peered":"2023-12-16T21:42:41.365758+0000","last_unstale":"2023-12-16T21:42:42.476620+0000","last_undegraded":"2023-12-16T21:42:42.476620+0000","last_fullsized":"2023-12-16T21:42:42.476620+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:17:38.230435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471210+0000","last_change":"2023-12-16T21:42:41.371042+0000","last_active":"2023-12-16T21:42:42.471210+0000","last_peered":"2023-12-16T21:42:42.471210+0000","last_clean":"2023-12-16T21:42:42.471210+0000","last_became_active":"2023-12-16T21:42:41.370204+0000","last_became_peered":"2023-12-16T21:42:41.370204+0000","last_unstale":"2023-12-16T21:42:42.471210+0000","last_undegraded":"2023-12-16T21:42:42.471210+0000","last_fullsized":"2023-12-16T21:42:42.471210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:21.194129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335291+0000","last_change":"2023-12-16T21:42:41.360173+0000","last_active":"2023-12-16T21:42:42.335291+0000","last_peered":"2023-12-16T21:42:42.335291+0000","last_clean":"2023-12-16T21:42:42.335291+0000","last_became_active":"2023-12-16T21:42:41.359373+0000","last_became_peered":"2023-12-16T21:42:41.359373+0000","last_unstale":"2023-12-16T21:42:42.335291+0000","last_undegraded":"2023-12-16T21:42:42.335291+0000","last_fullsized":"2023-12-16T21:42:42.335291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:59:44.179472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475288+0000","last_change":"2023-12-16T21:42:41.365867+0000","last_active":"2023-12-16T21:42:42.475288+0000","last_peered":"2023-12-16T21:42:42.475288+0000","last_clean":"2023-12-16T21:42:42.475288+0000","last_became_active":"2023-12-16T21:42:41.365202+0000","last_became_peered":"2023-12-16T21:42:41.365202+0000","last_unstale":"2023-12-16T21:42:42.475288+0000","last_undegraded":"2023-12-16T21:42:42.475288+0000","last_fullsized":"2023-12-16T21:42:42.475288+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471522+0000","last_change":"2023-12-16T21:42:41.377468+0000","last_active":"2023-12-16T21:42:42.471522+0000","last_peered":"2023-12-16T21:42:42.471522+0000","last_clean":"2023-12-16T21:42:42.471522+0000","last_became_active":"2023-12-16T21:42:41.376856+0000","last_became_peered":"2023-12-16T21:42:41.376856+0000","last_unstale":"2023-12-16T21:42:42.471522+0000","last_undegraded":"2023-12-16T21:42:42.471522+0000","last_fullsized":"2023-12-16T21:42:42.471522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:55.715961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475667+0000","last_change":"2023-12-16T21:42:41.366088+0000","last_active":"2023-12-16T21:42:42.475667+0000","last_peered":"2023-12-16T21:42:42.475667+0000","last_clean":"2023-12-16T21:42:42.475667+0000","last_became_active":"2023-12-16T21:42:41.365380+0000","last_became_peered":"2023-12-16T21:42:41.365380+0000","last_unstale":"2023-12-16T21:42:42.475667+0000","last_undegraded":"2023-12-16T21:42:42.475667+0000","last_fullsized":"2023-12-16T21:42:42.475667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:34.471133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471681+0000","last_change":"2023-12-16T21:42:41.378284+0000","last_active":"2023-12-16T21:42:42.471681+0000","last_peered":"2023-12-16T21:42:42.471681+0000","last_clean":"2023-12-16T21:42:42.471681+0000","last_became_active":"2023-12-16T21:42:41.377070+0000","last_became_peered":"2023-12-16T21:42:41.377070+0000","last_unstale":"2023-12-16T21:42:42.471681+0000","last_undegraded":"2023-12-16T21:42:42.471681+0000","last_fullsized":"2023-12-16T21:42:42.471681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475272+0000","last_change":"2023-12-16T21:42:41.376780+0000","last_active":"2023-12-16T21:42:42.475272+0000","last_peered":"2023-12-16T21:42:42.475272+0000","last_clean":"2023-12-16T21:42:42.475272+0000","last_became_active":"2023-12-16T21:42:41.376224+0000","last_became_peered":"2023-12-16T21:42:41.376224+0000","last_unstale":"2023-12-16T21:42:42.475272+0000","last_undegraded":"2023-12-16T21:42:42.475272+0000","last_fullsized":"2023-12-16T21:42:42.475272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472816+0000","last_change":"2023-12-16T21:42:41.372281+0000","last_active":"2023-12-16T21:42:42.472816+0000","last_peered":"2023-12-16T21:42:42.472816+0000","last_clean":"2023-12-16T21:42:42.472816+0000","last_became_active":"2023-12-16T21:42:41.371551+0000","last_became_peered":"2023-12-16T21:42:41.371551+0000","last_unstale":"2023-12-16T21:42:42.472816+0000","last_undegraded":"2023-12-16T21:42:42.472816+0000","last_fullsized":"2023-12-16T21:42:42.472816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475958+0000","last_change":"2023-12-16T21:42:41.375672+0000","last_active":"2023-12-16T21:42:42.475958+0000","last_peered":"2023-12-16T21:42:42.475958+0000","last_clean":"2023-12-16T21:42:42.475958+0000","last_became_active":"2023-12-16T21:42:41.370826+0000","last_became_peered":"2023-12-16T21:42:41.370826+0000","last_unstale":"2023-12-16T21:42:42.475958+0000","last_undegraded":"2023-12-16T21:42:42.475958+0000","last_fullsized":"2023-12-16T21:42:42.475958+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329872+0000","last_change":"2023-12-16T21:42:41.368001+0000","last_active":"2023-12-16T21:42:42.329872+0000","last_peered":"2023-12-16T21:42:42.329872+0000","last_clean":"2023-12-16T21:42:42.329872+0000","last_became_active":"2023-12-16T21:42:41.367168+0000","last_became_peered":"2023-12-16T21:42:41.367168+0000","last_unstale":"2023-12-16T21:42:42.329872+0000","last_undegraded":"2023-12-16T21:42:42.329872+0000","last_fullsized":"2023-12-16T21:42:42.329872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472910+0000","last_change":"2023-12-16T21:42:41.377845+0000","last_active":"2023-12-16T21:42:42.472910+0000","last_peered":"2023-12-16T21:42:42.472910+0000","last_clean":"2023-12-16T21:42:42.472910+0000","last_became_active":"2023-12-16T21:42:41.377023+0000","last_became_peered":"2023-12-16T21:42:41.377023+0000","last_unstale":"2023-12-16T21:42:42.472910+0000","last_undegraded":"2023-12-16T21:42:42.472910+0000","last_fullsized":"2023-12-16T21:42:42.472910+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472206+0000","last_change":"2023-12-16T21:42:41.371637+0000","last_active":"2023-12-16T21:42:42.472206+0000","last_peered":"2023-12-16T21:42:42.472206+0000","last_clean":"2023-12-16T21:42:42.472206+0000","last_became_active":"2023-12-16T21:42:41.370853+0000","last_became_peered":"2023-12-16T21:42:41.370853+0000","last_unstale":"2023-12-16T21:42:42.472206+0000","last_undegraded":"2023-12-16T21:42:42.472206+0000","last_fullsized":"2023-12-16T21:42:42.472206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475286+0000","last_change":"2023-12-16T21:42:41.375235+0000","last_active":"2023-12-16T21:42:42.475286+0000","last_peered":"2023-12-16T21:42:42.475286+0000","last_clean":"2023-12-16T21:42:42.475286+0000","last_became_active":"2023-12-16T21:42:41.374570+0000","last_became_peered":"2023-12-16T21:42:41.374570+0000","last_unstale":"2023-12-16T21:42:42.475286+0000","last_undegraded":"2023-12-16T21:42:42.475286+0000","last_fullsized":"2023-12-16T21:42:42.475286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472367+0000","last_change":"2023-12-16T21:42:41.383749+0000","last_active":"2023-12-16T21:42:42.472367+0000","last_peered":"2023-12-16T21:42:42.472367+0000","last_clean":"2023-12-16T21:42:42.472367+0000","last_became_active":"2023-12-16T21:42:41.383009+0000","last_became_peered":"2023-12-16T21:42:41.383009+0000","last_unstale":"2023-12-16T21:42:42.472367+0000","last_undegraded":"2023-12-16T21:42:42.472367+0000","last_fullsized":"2023-12-16T21:42:42.472367+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473979+0000","last_change":"2023-12-16T21:42:41.379801+0000","last_active":"2023-12-16T21:42:42.473979+0000","last_peered":"2023-12-16T21:42:42.473979+0000","last_clean":"2023-12-16T21:42:42.473979+0000","last_became_active":"2023-12-16T21:42:41.379000+0000","last_became_peered":"2023-12-16T21:42:41.379000+0000","last_unstale":"2023-12-16T21:42:42.473979+0000","last_undegraded":"2023-12-16T21:42:42.473979+0000","last_fullsized":"2023-12-16T21:42:42.473979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333784+0000","last_change":"2023-12-16T21:42:41.351207+0000","last_active":"2023-12-16T21:42:42.333784+0000","last_peered":"2023-12-16T21:42:42.333784+0000","last_clean":"2023-12-16T21:42:42.333784+0000","last_became_active":"2023-12-16T21:42:41.349156+0000","last_became_peered":"2023-12-16T21:42:41.349156+0000","last_unstale":"2023-12-16T21:42:42.333784+0000","last_undegraded":"2023-12-16T21:42:42.333784+0000","last_fullsized":"2023-12-16T21:42:42.333784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:52.130538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472650+0000","last_change":"2023-12-16T21:42:40.442950+0000","last_active":"2023-12-16T21:42:42.472650+0000","last_peered":"2023-12-16T21:42:42.472650+0000","last_clean":"2023-12-16T21:42:42.472650+0000","last_became_active":"2023-12-16T21:42:40.442532+0000","last_became_peered":"2023-12-16T21:42:40.442532+0000","last_unstale":"2023-12-16T21:42:42.472650+0000","last_undegraded":"2023-12-16T21:42:42.472650+0000","last_fullsized":"2023-12-16T21:42:42.472650+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472291+0000","last_change":"2023-12-16T21:42:41.384406+0000","last_active":"2023-12-16T21:42:42.472291+0000","last_peered":"2023-12-16T21:42:42.472291+0000","last_clean":"2023-12-16T21:42:42.472291+0000","last_became_active":"2023-12-16T21:42:41.383814+0000","last_became_peered":"2023-12-16T21:42:41.383814+0000","last_unstale":"2023-12-16T21:42:42.472291+0000","last_undegraded":"2023-12-16T21:42:42.472291+0000","last_fullsized":"2023-12-16T21:42:42.472291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475081+0000","last_change":"2023-12-16T21:42:40.437381+0000","last_active":"2023-12-16T21:42:42.475081+0000","last_peered":"2023-12-16T21:42:42.475081+0000","last_clean":"2023-12-16T21:42:42.475081+0000","last_became_active":"2023-12-16T21:42:40.436469+0000","last_became_peered":"2023-12-16T21:42:40.436469+0000","last_unstale":"2023-12-16T21:42:42.475081+0000","last_undegraded":"2023-12-16T21:42:42.475081+0000","last_fullsized":"2023-12-16T21:42:42.475081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471777+0000","last_change":"2023-12-16T21:42:40.432572+0000","last_active":"2023-12-16T21:42:42.471777+0000","last_peered":"2023-12-16T21:42:42.471777+0000","last_clean":"2023-12-16T21:42:42.471777+0000","last_became_active":"2023-12-16T21:42:40.431973+0000","last_became_peered":"2023-12-16T21:42:40.431973+0000","last_unstale":"2023-12-16T21:42:42.471777+0000","last_undegraded":"2023-12-16T21:42:42.471777+0000","last_fullsized":"2023-12-16T21:42:42.471777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472032+0000","last_change":"2023-12-16T21:42:41.372173+0000","last_active":"2023-12-16T21:42:42.472032+0000","last_peered":"2023-12-16T21:42:42.472032+0000","last_clean":"2023-12-16T21:42:42.472032+0000","last_became_active":"2023-12-16T21:42:41.371362+0000","last_became_peered":"2023-12-16T21:42:41.371362+0000","last_unstale":"2023-12-16T21:42:42.472032+0000","last_undegraded":"2023-12-16T21:42:42.472032+0000","last_fullsized":"2023-12-16T21:42:42.472032+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474354+0000","last_change":"2023-12-16T21:42:40.458568+0000","last_active":"2023-12-16T21:42:42.474354+0000","last_peered":"2023-12-16T21:42:42.474354+0000","last_clean":"2023-12-16T21:42:42.474354+0000","last_became_active":"2023-12-16T21:42:40.457780+0000","last_became_peered":"2023-12-16T21:42:40.457780+0000","last_unstale":"2023-12-16T21:42:42.474354+0000","last_undegraded":"2023-12-16T21:42:42.474354+0000","last_fullsized":"2023-12-16T21:42:42.474354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329203+0000","last_change":"2023-12-16T21:42:41.367605+0000","last_active":"2023-12-16T21:42:42.329203+0000","last_peered":"2023-12-16T21:42:42.329203+0000","last_clean":"2023-12-16T21:42:42.329203+0000","last_became_active":"2023-12-16T21:42:41.366781+0000","last_became_peered":"2023-12-16T21:42:41.366781+0000","last_unstale":"2023-12-16T21:42:42.329203+0000","last_undegraded":"2023-12-16T21:42:42.329203+0000","last_fullsized":"2023-12-16T21:42:42.329203+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475185+0000","last_change":"2023-12-16T21:42:40.462161+0000","last_active":"2023-12-16T21:42:42.475185+0000","last_peered":"2023-12-16T21:42:42.475185+0000","last_clean":"2023-12-16T21:42:42.475185+0000","last_became_active":"2023-12-16T21:42:40.461535+0000","last_became_peered":"2023-12-16T21:42:40.461535+0000","last_unstale":"2023-12-16T21:42:42.475185+0000","last_undegraded":"2023-12-16T21:42:42.475185+0000","last_fullsized":"2023-12-16T21:42:42.475185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476429+0000","last_change":"2023-12-16T21:42:41.366419+0000","last_active":"2023-12-16T21:42:42.476429+0000","last_peered":"2023-12-16T21:42:42.476429+0000","last_clean":"2023-12-16T21:42:42.476429+0000","last_became_active":"2023-12-16T21:42:41.365644+0000","last_became_peered":"2023-12-16T21:42:41.365644+0000","last_unstale":"2023-12-16T21:42:42.476429+0000","last_undegraded":"2023-12-16T21:42:42.476429+0000","last_fullsized":"2023-12-16T21:42:42.476429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476308+0000","last_change":"2023-12-16T21:42:40.461870+0000","last_active":"2023-12-16T21:42:42.476308+0000","last_peered":"2023-12-16T21:42:42.476308+0000","last_clean":"2023-12-16T21:42:42.476308+0000","last_became_active":"2023-12-16T21:42:40.461189+0000","last_became_peered":"2023-12-16T21:42:40.461189+0000","last_unstale":"2023-12-16T21:42:42.476308+0000","last_undegraded":"2023-12-16T21:42:42.476308+0000","last_fullsized":"2023-12-16T21:42:42.476308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471926+0000","last_change":"2023-12-16T21:42:41.385522+0000","last_active":"2023-12-16T21:42:42.471926+0000","last_peered":"2023-12-16T21:42:42.471926+0000","last_clean":"2023-12-16T21:42:42.471926+0000","last_became_active":"2023-12-16T21:42:41.384952+0000","last_became_peered":"2023-12-16T21:42:41.384952+0000","last_unstale":"2023-12-16T21:42:42.471926+0000","last_undegraded":"2023-12-16T21:42:42.471926+0000","last_fullsized":"2023-12-16T21:42:42.471926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330102+0000","last_change":"2023-12-16T21:42:40.453133+0000","last_active":"2023-12-16T21:42:42.330102+0000","last_peered":"2023-12-16T21:42:42.330102+0000","last_clean":"2023-12-16T21:42:42.330102+0000","last_became_active":"2023-12-16T21:42:40.452814+0000","last_became_peered":"2023-12-16T21:42:40.452814+0000","last_unstale":"2023-12-16T21:42:42.330102+0000","last_undegraded":"2023-12-16T21:42:42.330102+0000","last_fullsized":"2023-12-16T21:42:42.330102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335567+0000","last_change":"2023-12-16T21:42:41.359165+0000","last_active":"2023-12-16T21:42:42.335567+0000","last_peered":"2023-12-16T21:42:42.335567+0000","last_clean":"2023-12-16T21:42:42.335567+0000","last_became_active":"2023-12-16T21:42:41.356738+0000","last_became_peered":"2023-12-16T21:42:41.356738+0000","last_unstale":"2023-12-16T21:42:42.335567+0000","last_undegraded":"2023-12-16T21:42:42.335567+0000","last_fullsized":"2023-12-16T21:42:42.335567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:13:31.197727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475530+0000","last_change":"2023-12-16T21:42:40.428771+0000","last_active":"2023-12-16T21:42:42.475530+0000","last_peered":"2023-12-16T21:42:42.475530+0000","last_clean":"2023-12-16T21:42:42.475530+0000","last_became_active":"2023-12-16T21:42:40.427921+0000","last_became_peered":"2023-12-16T21:42:40.427921+0000","last_unstale":"2023-12-16T21:42:42.475530+0000","last_undegraded":"2023-12-16T21:42:42.475530+0000","last_fullsized":"2023-12-16T21:42:42.475530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:41.800567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334711+0000","last_change":"2023-12-16T21:42:41.373637+0000","last_active":"2023-12-16T21:42:42.334711+0000","last_peered":"2023-12-16T21:42:42.334711+0000","last_clean":"2023-12-16T21:42:42.334711+0000","last_became_active":"2023-12-16T21:42:41.373003+0000","last_became_peered":"2023-12-16T21:42:41.373003+0000","last_unstale":"2023-12-16T21:42:42.334711+0000","last_undegraded":"2023-12-16T21:42:42.334711+0000","last_fullsized":"2023-12-16T21:42:42.334711+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:48:55.034570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472336+0000","last_change":"2023-12-16T21:42:40.424774+0000","last_active":"2023-12-16T21:42:42.472336+0000","last_peered":"2023-12-16T21:42:42.472336+0000","last_clean":"2023-12-16T21:42:42.472336+0000","last_became_active":"2023-12-16T21:42:40.424333+0000","last_became_peered":"2023-12-16T21:42:40.424333+0000","last_unstale":"2023-12-16T21:42:42.472336+0000","last_undegraded":"2023-12-16T21:42:42.472336+0000","last_fullsized":"2023-12-16T21:42:42.472336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472450+0000","last_change":"2023-12-16T21:42:41.377224+0000","last_active":"2023-12-16T21:42:42.472450+0000","last_peered":"2023-12-16T21:42:42.472450+0000","last_clean":"2023-12-16T21:42:42.472450+0000","last_became_active":"2023-12-16T21:42:41.376574+0000","last_became_peered":"2023-12-16T21:42:41.376574+0000","last_unstale":"2023-12-16T21:42:42.472450+0000","last_undegraded":"2023-12-16T21:42:42.472450+0000","last_fullsized":"2023-12-16T21:42:42.472450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:39.848468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475529+0000","last_change":"2023-12-16T21:42:40.453599+0000","last_active":"2023-12-16T21:42:42.475529+0000","last_peered":"2023-12-16T21:42:42.475529+0000","last_clean":"2023-12-16T21:42:42.475529+0000","last_became_active":"2023-12-16T21:42:40.452662+0000","last_became_peered":"2023-12-16T21:42:40.452662+0000","last_unstale":"2023-12-16T21:42:42.475529+0000","last_undegraded":"2023-12-16T21:42:42.475529+0000","last_fullsized":"2023-12-16T21:42:42.475529+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:02.018716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.333919+0000","last_change":"2023-12-16T21:42:41.365750+0000","last_active":"2023-12-16T21:42:42.333919+0000","last_peered":"2023-12-16T21:42:42.333919+0000","last_clean":"2023-12-16T21:42:42.333919+0000","last_became_active":"2023-12-16T21:42:41.365097+0000","last_became_peered":"2023-12-16T21:42:41.365097+0000","last_unstale":"2023-12-16T21:42:42.333919+0000","last_undegraded":"2023-12-16T21:42:42.333919+0000","last_fullsized":"2023-12-16T21:42:42.333919+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:04:18.297735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474655+0000","last_change":"2023-12-16T21:42:40.437256+0000","last_active":"2023-12-16T21:42:42.474655+0000","last_peered":"2023-12-16T21:42:42.474655+0000","last_clean":"2023-12-16T21:42:42.474655+0000","last_became_active":"2023-12-16T21:42:40.436394+0000","last_became_peered":"2023-12-16T21:42:40.436394+0000","last_unstale":"2023-12-16T21:42:42.474655+0000","last_undegraded":"2023-12-16T21:42:42.474655+0000","last_fullsized":"2023-12-16T21:42:42.474655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.473943+0000","last_change":"2023-12-16T21:42:41.373882+0000","last_active":"2023-12-16T21:42:42.473943+0000","last_peered":"2023-12-16T21:42:42.473943+0000","last_clean":"2023-12-16T21:42:42.473943+0000","last_became_active":"2023-12-16T21:42:41.373250+0000","last_became_peered":"2023-12-16T21:42:41.373250+0000","last_unstale":"2023-12-16T21:42:42.473943+0000","last_undegraded":"2023-12-16T21:42:42.473943+0000","last_fullsized":"2023-12-16T21:42:42.473943+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471379+0000","last_change":"2023-12-16T21:42:40.428292+0000","last_active":"2023-12-16T21:42:42.471379+0000","last_peered":"2023-12-16T21:42:42.471379+0000","last_clean":"2023-12-16T21:42:42.471379+0000","last_became_active":"2023-12-16T21:42:40.427611+0000","last_became_peered":"2023-12-16T21:42:40.427611+0000","last_unstale":"2023-12-16T21:42:42.471379+0000","last_undegraded":"2023-12-16T21:42:42.471379+0000","last_fullsized":"2023-12-16T21:42:42.471379+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:56:15.580038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332822+0000","last_change":"2023-12-16T21:42:41.364395+0000","last_active":"2023-12-16T21:42:42.332822+0000","last_peered":"2023-12-16T21:42:42.332822+0000","last_clean":"2023-12-16T21:42:42.332822+0000","last_became_active":"2023-12-16T21:42:41.363596+0000","last_became_peered":"2023-12-16T21:42:41.363596+0000","last_unstale":"2023-12-16T21:42:42.332822+0000","last_undegraded":"2023-12-16T21:42:42.332822+0000","last_fullsized":"2023-12-16T21:42:42.332822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:07:23.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335857+0000","last_change":"2023-12-16T21:42:40.430896+0000","last_active":"2023-12-16T21:42:42.335857+0000","last_peered":"2023-12-16T21:42:42.335857+0000","last_clean":"2023-12-16T21:42:42.335857+0000","last_became_active":"2023-12-16T21:42:40.425814+0000","last_became_peered":"2023-12-16T21:42:40.425814+0000","last_unstale":"2023-12-16T21:42:42.335857+0000","last_undegraded":"2023-12-16T21:42:42.335857+0000","last_fullsized":"2023-12-16T21:42:42.335857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:39:34.410203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334498+0000","last_change":"2023-12-16T21:42:41.364963+0000","last_active":"2023-12-16T21:42:42.334498+0000","last_peered":"2023-12-16T21:42:42.334498+0000","last_clean":"2023-12-16T21:42:42.334498+0000","last_became_active":"2023-12-16T21:42:41.364207+0000","last_became_peered":"2023-12-16T21:42:41.364207+0000","last_unstale":"2023-12-16T21:42:42.334498+0000","last_undegraded":"2023-12-16T21:42:42.334498+0000","last_fullsized":"2023-12-16T21:42:42.334498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:11:32.433518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471333+0000","last_change":"2023-12-16T21:42:40.424410+0000","last_active":"2023-12-16T21:42:42.471333+0000","last_peered":"2023-12-16T21:42:42.471333+0000","last_clean":"2023-12-16T21:42:42.471333+0000","last_became_active":"2023-12-16T21:42:40.423891+0000","last_became_peered":"2023-12-16T21:42:40.423891+0000","last_unstale":"2023-12-16T21:42:42.471333+0000","last_undegraded":"2023-12-16T21:42:42.471333+0000","last_fullsized":"2023-12-16T21:42:42.471333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334300+0000","last_change":"2023-12-16T21:42:41.366442+0000","last_active":"2023-12-16T21:42:42.334300+0000","last_peered":"2023-12-16T21:42:42.334300+0000","last_clean":"2023-12-16T21:42:42.334300+0000","last_became_active":"2023-12-16T21:42:41.363759+0000","last_became_peered":"2023-12-16T21:42:41.363759+0000","last_unstale":"2023-12-16T21:42:42.334300+0000","last_undegraded":"2023-12-16T21:42:42.334300+0000","last_fullsized":"2023-12-16T21:42:42.334300+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:44:20.523567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.330332+0000","last_change":"2023-12-16T21:42:40.429084+0000","last_active":"2023-12-16T21:42:42.330332+0000","last_peered":"2023-12-16T21:42:42.330332+0000","last_clean":"2023-12-16T21:42:42.330332+0000","last_became_active":"2023-12-16T21:42:40.428576+0000","last_became_peered":"2023-12-16T21:42:40.428576+0000","last_unstale":"2023-12-16T21:42:42.330332+0000","last_undegraded":"2023-12-16T21:42:42.330332+0000","last_fullsized":"2023-12-16T21:42:42.330332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:10:54.704020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475041+0000","last_change":"2023-12-16T21:42:41.385053+0000","last_active":"2023-12-16T21:42:42.475041+0000","last_peered":"2023-12-16T21:42:42.475041+0000","last_clean":"2023-12-16T21:42:42.475041+0000","last_became_active":"2023-12-16T21:42:41.384196+0000","last_became_peered":"2023-12-16T21:42:41.384196+0000","last_unstale":"2023-12-16T21:42:42.475041+0000","last_undegraded":"2023-12-16T21:42:42.475041+0000","last_fullsized":"2023-12-16T21:42:42.475041+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:35:34.296899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471297+0000","last_change":"2023-12-16T21:42:40.461203+0000","last_active":"2023-12-16T21:42:42.471297+0000","last_peered":"2023-12-16T21:42:42.471297+0000","last_clean":"2023-12-16T21:42:42.471297+0000","last_became_active":"2023-12-16T21:42:40.460912+0000","last_became_peered":"2023-12-16T21:42:40.460912+0000","last_unstale":"2023-12-16T21:42:42.471297+0000","last_undegraded":"2023-12-16T21:42:42.471297+0000","last_fullsized":"2023-12-16T21:42:42.471297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:30:51.118596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335069+0000","last_change":"2023-12-16T21:42:41.369186+0000","last_active":"2023-12-16T21:42:42.335069+0000","last_peered":"2023-12-16T21:42:42.335069+0000","last_clean":"2023-12-16T21:42:42.335069+0000","last_became_active":"2023-12-16T21:42:41.368337+0000","last_became_peered":"2023-12-16T21:42:41.368337+0000","last_unstale":"2023-12-16T21:42:42.335069+0000","last_undegraded":"2023-12-16T21:42:42.335069+0000","last_fullsized":"2023-12-16T21:42:42.335069+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:57:56.730962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-16T21:42:41.331102+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:42:41.331102+0000","last_peered":"2023-12-16T21:42:41.331102+0000","last_clean":"2023-12-16T21:42:41.331102+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:42:41.331102+0000","last_undegraded":"2023-12-16T21:42:41.331102+0000","last_fullsized":"2023-12-16T21:42:41.331102+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474341+0000","last_change":"2023-12-16T21:42:40.457329+0000","last_active":"2023-12-16T21:42:42.474341+0000","last_peered":"2023-12-16T21:42:42.474341+0000","last_clean":"2023-12-16T21:42:42.474341+0000","last_became_active":"2023-12-16T21:42:40.453943+0000","last_became_peered":"2023-12-16T21:42:40.453943+0000","last_unstale":"2023-12-16T21:42:42.474341+0000","last_undegraded":"2023-12-16T21:42:42.474341+0000","last_fullsized":"2023-12-16T21:42:42.474341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:06:05.409461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472035+0000","last_change":"2023-12-16T21:42:41.367983+0000","last_active":"2023-12-16T21:42:42.472035+0000","last_peered":"2023-12-16T21:42:42.472035+0000","last_clean":"2023-12-16T21:42:42.472035+0000","last_became_active":"2023-12-16T21:42:41.367201+0000","last_became_peered":"2023-12-16T21:42:41.367201+0000","last_unstale":"2023-12-16T21:42:42.472035+0000","last_undegraded":"2023-12-16T21:42:42.472035+0000","last_fullsized":"2023-12-16T21:42:42.472035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332137+0000","last_change":"2023-12-16T21:42:40.447913+0000","last_active":"2023-12-16T21:42:42.332137+0000","last_peered":"2023-12-16T21:42:42.332137+0000","last_clean":"2023-12-16T21:42:42.332137+0000","last_became_active":"2023-12-16T21:42:40.445110+0000","last_became_peered":"2023-12-16T21:42:40.445110+0000","last_unstale":"2023-12-16T21:42:42.332137+0000","last_undegraded":"2023-12-16T21:42:42.332137+0000","last_fullsized":"2023-12-16T21:42:42.332137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471511+0000","last_change":"2023-12-16T21:42:41.378992+0000","last_active":"2023-12-16T21:42:42.471511+0000","last_peered":"2023-12-16T21:42:42.471511+0000","last_clean":"2023-12-16T21:42:42.471511+0000","last_became_active":"2023-12-16T21:42:41.378192+0000","last_became_peered":"2023-12-16T21:42:41.378192+0000","last_unstale":"2023-12-16T21:42:42.471511+0000","last_undegraded":"2023-12-16T21:42:42.471511+0000","last_fullsized":"2023-12-16T21:42:42.471511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:32:42.812508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472434+0000","last_change":"2023-12-16T21:42:40.424908+0000","last_active":"2023-12-16T21:42:42.472434+0000","last_peered":"2023-12-16T21:42:42.472434+0000","last_clean":"2023-12-16T21:42:42.472434+0000","last_became_active":"2023-12-16T21:42:40.424358+0000","last_became_peered":"2023-12-16T21:42:40.424358+0000","last_unstale":"2023-12-16T21:42:42.472434+0000","last_undegraded":"2023-12-16T21:42:42.472434+0000","last_fullsized":"2023-12-16T21:42:42.472434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329469+0000","last_change":"2023-12-16T21:42:41.352792+0000","last_active":"2023-12-16T21:42:42.329469+0000","last_peered":"2023-12-16T21:42:42.329469+0000","last_clean":"2023-12-16T21:42:42.329469+0000","last_became_active":"2023-12-16T21:42:41.352078+0000","last_became_peered":"2023-12-16T21:42:41.352078+0000","last_unstale":"2023-12-16T21:42:42.329469+0000","last_undegraded":"2023-12-16T21:42:42.329469+0000","last_fullsized":"2023-12-16T21:42:42.329469+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:58.952795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475741+0000","last_change":"2023-12-16T21:42:40.457237+0000","last_active":"2023-12-16T21:42:42.475741+0000","last_peered":"2023-12-16T21:42:42.475741+0000","last_clean":"2023-12-16T21:42:42.475741+0000","last_became_active":"2023-12-16T21:42:40.452459+0000","last_became_peered":"2023-12-16T21:42:40.452459+0000","last_unstale":"2023-12-16T21:42:42.475741+0000","last_undegraded":"2023-12-16T21:42:42.475741+0000","last_fullsized":"2023-12-16T21:42:42.475741+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:19:57.940854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334555+0000","last_change":"2023-12-16T21:42:41.352263+0000","last_active":"2023-12-16T21:42:42.334555+0000","last_peered":"2023-12-16T21:42:42.334555+0000","last_clean":"2023-12-16T21:42:42.334555+0000","last_became_active":"2023-12-16T21:42:41.347293+0000","last_became_peered":"2023-12-16T21:42:41.347293+0000","last_unstale":"2023-12-16T21:42:42.334555+0000","last_undegraded":"2023-12-16T21:42:42.334555+0000","last_fullsized":"2023-12-16T21:42:42.334555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:52.699335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329677+0000","last_change":"2023-12-16T21:42:40.423235+0000","last_active":"2023-12-16T21:42:42.329677+0000","last_peered":"2023-12-16T21:42:42.329677+0000","last_clean":"2023-12-16T21:42:42.329677+0000","last_became_active":"2023-12-16T21:42:40.422630+0000","last_became_peered":"2023-12-16T21:42:40.422630+0000","last_unstale":"2023-12-16T21:42:42.329677+0000","last_undegraded":"2023-12-16T21:42:42.329677+0000","last_fullsized":"2023-12-16T21:42:42.329677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475779+0000","last_change":"2023-12-16T21:42:41.372994+0000","last_active":"2023-12-16T21:42:42.475779+0000","last_peered":"2023-12-16T21:42:42.475779+0000","last_clean":"2023-12-16T21:42:42.475779+0000","last_became_active":"2023-12-16T21:42:41.372135+0000","last_became_peered":"2023-12-16T21:42:41.372135+0000","last_unstale":"2023-12-16T21:42:42.475779+0000","last_undegraded":"2023-12-16T21:42:42.475779+0000","last_fullsized":"2023-12-16T21:42:42.475779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:50:50.069721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335404+0000","last_change":"2023-12-16T21:42:40.445536+0000","last_active":"2023-12-16T21:42:42.335404+0000","last_peered":"2023-12-16T21:42:42.335404+0000","last_clean":"2023-12-16T21:42:42.335404+0000","last_became_active":"2023-12-16T21:42:40.444652+0000","last_became_peered":"2023-12-16T21:42:40.444652+0000","last_unstale":"2023-12-16T21:42:42.335404+0000","last_undegraded":"2023-12-16T21:42:42.335404+0000","last_fullsized":"2023-12-16T21:42:42.335404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:55:38.836527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329548+0000","last_change":"2023-12-16T21:42:41.352754+0000","last_active":"2023-12-16T21:42:42.329548+0000","last_peered":"2023-12-16T21:42:42.329548+0000","last_clean":"2023-12-16T21:42:42.329548+0000","last_became_active":"2023-12-16T21:42:41.351903+0000","last_became_peered":"2023-12-16T21:42:41.351903+0000","last_unstale":"2023-12-16T21:42:42.329548+0000","last_undegraded":"2023-12-16T21:42:42.329548+0000","last_fullsized":"2023-12-16T21:42:42.329548+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:21.434797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.687428+0000","last_change":"2023-12-16T21:42:40.425139+0000","last_active":"2023-12-16T21:42:42.687428+0000","last_peered":"2023-12-16T21:42:42.687428+0000","last_clean":"2023-12-16T21:42:42.687428+0000","last_became_active":"2023-12-16T21:42:40.423820+0000","last_became_peered":"2023-12-16T21:42:40.423820+0000","last_unstale":"2023-12-16T21:42:42.687428+0000","last_undegraded":"2023-12-16T21:42:42.687428+0000","last_fullsized":"2023-12-16T21:42:42.687428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:37.970083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472944+0000","last_change":"2023-12-16T21:42:41.365965+0000","last_active":"2023-12-16T21:42:42.472944+0000","last_peered":"2023-12-16T21:42:42.472944+0000","last_clean":"2023-12-16T21:42:42.472944+0000","last_became_active":"2023-12-16T21:42:41.365191+0000","last_became_peered":"2023-12-16T21:42:41.365191+0000","last_unstale":"2023-12-16T21:42:42.472944+0000","last_undegraded":"2023-12-16T21:42:42.472944+0000","last_fullsized":"2023-12-16T21:42:42.472944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.476214+0000","last_change":"2023-12-16T21:42:40.447186+0000","last_active":"2023-12-16T21:42:42.476214+0000","last_peered":"2023-12-16T21:42:42.476214+0000","last_clean":"2023-12-16T21:42:42.476214+0000","last_became_active":"2023-12-16T21:42:40.446374+0000","last_became_peered":"2023-12-16T21:42:40.446374+0000","last_unstale":"2023-12-16T21:42:42.476214+0000","last_undegraded":"2023-12-16T21:42:42.476214+0000","last_fullsized":"2023-12-16T21:42:42.476214+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471782+0000","last_change":"2023-12-16T21:42:41.364750+0000","last_active":"2023-12-16T21:42:42.471782+0000","last_peered":"2023-12-16T21:42:42.471782+0000","last_clean":"2023-12-16T21:42:42.471782+0000","last_became_active":"2023-12-16T21:42:41.363976+0000","last_became_peered":"2023-12-16T21:42:41.363976+0000","last_unstale":"2023-12-16T21:42:42.471782+0000","last_undegraded":"2023-12-16T21:42:42.471782+0000","last_fullsized":"2023-12-16T21:42:42.471782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:27.202820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332140+0000","last_change":"2023-12-16T21:42:40.444481+0000","last_active":"2023-12-16T21:42:42.332140+0000","last_peered":"2023-12-16T21:42:42.332140+0000","last_clean":"2023-12-16T21:42:42.332140+0000","last_became_active":"2023-12-16T21:42:40.443665+0000","last_became_peered":"2023-12-16T21:42:40.443665+0000","last_unstale":"2023-12-16T21:42:42.332140+0000","last_undegraded":"2023-12-16T21:42:42.332140+0000","last_fullsized":"2023-12-16T21:42:42.332140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:25:51.687393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334181+0000","last_change":"2023-12-16T21:42:41.360039+0000","last_active":"2023-12-16T21:42:42.334181+0000","last_peered":"2023-12-16T21:42:42.334181+0000","last_clean":"2023-12-16T21:42:42.334181+0000","last_became_active":"2023-12-16T21:42:41.359132+0000","last_became_peered":"2023-12-16T21:42:41.359132+0000","last_unstale":"2023-12-16T21:42:42.334181+0000","last_undegraded":"2023-12-16T21:42:42.334181+0000","last_fullsized":"2023-12-16T21:42:42.334181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:52.703815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329301+0000","last_change":"2023-12-16T21:42:40.438910+0000","last_active":"2023-12-16T21:42:42.329301+0000","last_peered":"2023-12-16T21:42:42.329301+0000","last_clean":"2023-12-16T21:42:42.329301+0000","last_became_active":"2023-12-16T21:42:40.438462+0000","last_became_peered":"2023-12-16T21:42:40.438462+0000","last_unstale":"2023-12-16T21:42:42.329301+0000","last_undegraded":"2023-12-16T21:42:42.329301+0000","last_fullsized":"2023-12-16T21:42:42.329301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:21:34.818933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472184+0000","last_change":"2023-12-16T21:42:41.364881+0000","last_active":"2023-12-16T21:42:42.472184+0000","last_peered":"2023-12-16T21:42:42.472184+0000","last_clean":"2023-12-16T21:42:42.472184+0000","last_became_active":"2023-12-16T21:42:41.363545+0000","last_became_peered":"2023-12-16T21:42:41.363545+0000","last_unstale":"2023-12-16T21:42:42.472184+0000","last_undegraded":"2023-12-16T21:42:42.472184+0000","last_fullsized":"2023-12-16T21:42:42.472184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:54.173940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.329671+0000","last_change":"2023-12-16T21:42:40.418011+0000","last_active":"2023-12-16T21:42:42.329671+0000","last_peered":"2023-12-16T21:42:42.329671+0000","last_clean":"2023-12-16T21:42:42.329671+0000","last_became_active":"2023-12-16T21:42:40.417587+0000","last_became_peered":"2023-12-16T21:42:40.417587+0000","last_unstale":"2023-12-16T21:42:42.329671+0000","last_undegraded":"2023-12-16T21:42:42.329671+0000","last_fullsized":"2023-12-16T21:42:42.329671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:53:51.691874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472662+0000","last_change":"2023-12-16T21:42:41.380236+0000","last_active":"2023-12-16T21:42:42.472662+0000","last_peered":"2023-12-16T21:42:42.472662+0000","last_clean":"2023-12-16T21:42:42.472662+0000","last_became_active":"2023-12-16T21:42:41.379681+0000","last_became_peered":"2023-12-16T21:42:41.379681+0000","last_unstale":"2023-12-16T21:42:42.472662+0000","last_undegraded":"2023-12-16T21:42:42.472662+0000","last_fullsized":"2023-12-16T21:42:42.472662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:23:44.721396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.334158+0000","last_change":"2023-12-16T21:42:40.448985+0000","last_active":"2023-12-16T21:42:42.334158+0000","last_peered":"2023-12-16T21:42:42.334158+0000","last_clean":"2023-12-16T21:42:42.334158+0000","last_became_active":"2023-12-16T21:42:40.443742+0000","last_became_peered":"2023-12-16T21:42:40.443742+0000","last_unstale":"2023-12-16T21:42:42.334158+0000","last_undegraded":"2023-12-16T21:42:42.334158+0000","last_fullsized":"2023-12-16T21:42:42.334158+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:39:53.161999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474831+0000","last_change":"2023-12-16T21:42:41.363951+0000","last_active":"2023-12-16T21:42:42.474831+0000","last_peered":"2023-12-16T21:42:42.474831+0000","last_clean":"2023-12-16T21:42:42.474831+0000","last_became_active":"2023-12-16T21:42:41.363161+0000","last_became_peered":"2023-12-16T21:42:41.363161+0000","last_unstale":"2023-12-16T21:42:42.474831+0000","last_undegraded":"2023-12-16T21:42:42.474831+0000","last_fullsized":"2023-12-16T21:42:42.474831+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:01:03.030657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.475088+0000","last_change":"2023-12-16T21:42:41.386385+0000","last_active":"2023-12-16T21:42:42.475088+0000","last_peered":"2023-12-16T21:42:42.475088+0000","last_clean":"2023-12-16T21:42:42.475088+0000","last_became_active":"2023-12-16T21:42:41.385339+0000","last_became_peered":"2023-12-16T21:42:41.385339+0000","last_unstale":"2023-12-16T21:42:42.475088+0000","last_undegraded":"2023-12-16T21:42:42.475088+0000","last_fullsized":"2023-12-16T21:42:42.475088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:16:04.234197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.332799+0000","last_change":"2023-12-16T21:42:40.462751+0000","last_active":"2023-12-16T21:42:42.332799+0000","last_peered":"2023-12-16T21:42:42.332799+0000","last_clean":"2023-12-16T21:42:42.332799+0000","last_became_active":"2023-12-16T21:42:40.461936+0000","last_became_peered":"2023-12-16T21:42:40.461936+0000","last_unstale":"2023-12-16T21:42:42.332799+0000","last_undegraded":"2023-12-16T21:42:42.332799+0000","last_fullsized":"2023-12-16T21:42:42.332799+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.474683+0000","last_change":"2023-12-16T21:42:41.384715+0000","last_active":"2023-12-16T21:42:42.474683+0000","last_peered":"2023-12-16T21:42:42.474683+0000","last_clean":"2023-12-16T21:42:42.474683+0000","last_became_active":"2023-12-16T21:42:41.383596+0000","last_became_peered":"2023-12-16T21:42:41.383596+0000","last_unstale":"2023-12-16T21:42:42.474683+0000","last_undegraded":"2023-12-16T21:42:42.474683+0000","last_fullsized":"2023-12-16T21:42:42.474683+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.335044+0000","last_change":"2023-12-16T21:42:40.445218+0000","last_active":"2023-12-16T21:42:42.335044+0000","last_peered":"2023-12-16T21:42:42.335044+0000","last_clean":"2023-12-16T21:42:42.335044+0000","last_became_active":"2023-12-16T21:42:40.443892+0000","last_became_peered":"2023-12-16T21:42:40.443892+0000","last_unstale":"2023-12-16T21:42:42.335044+0000","last_undegraded":"2023-12-16T21:42:42.335044+0000","last_fullsized":"2023-12-16T21:42:42.335044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:02:20.422855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471754+0000","last_change":"2023-12-16T21:42:41.385409+0000","last_active":"2023-12-16T21:42:42.471754+0000","last_peered":"2023-12-16T21:42:42.471754+0000","last_clean":"2023-12-16T21:42:42.471754+0000","last_became_active":"2023-12-16T21:42:41.384925+0000","last_became_peered":"2023-12-16T21:42:41.384925+0000","last_unstale":"2023-12-16T21:42:42.471754+0000","last_undegraded":"2023-12-16T21:42:42.471754+0000","last_fullsized":"2023-12-16T21:42:42.471754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:36:38.982024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472993+0000","last_change":"2023-12-16T21:42:40.417361+0000","last_active":"2023-12-16T21:42:42.472993+0000","last_peered":"2023-12-16T21:42:42.472993+0000","last_clean":"2023-12-16T21:42:42.472993+0000","last_became_active":"2023-12-16T21:42:40.416759+0000","last_became_peered":"2023-12-16T21:42:40.416759+0000","last_unstale":"2023-12-16T21:42:42.472993+0000","last_undegraded":"2023-12-16T21:42:42.472993+0000","last_fullsized":"2023-12-16T21:42:42.472993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:45:20.182187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.472885+0000","last_change":"2023-12-16T21:42:41.370637+0000","last_active":"2023-12-16T21:42:42.472885+0000","last_peered":"2023-12-16T21:42:42.472885+0000","last_clean":"2023-12-16T21:42:42.472885+0000","last_became_active":"2023-12-16T21:42:41.369929+0000","last_became_peered":"2023-12-16T21:42:41.369929+0000","last_unstale":"2023-12-16T21:42:42.472885+0000","last_undegraded":"2023-12-16T21:42:42.472885+0000","last_fullsized":"2023-12-16T21:42:42.472885+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:40.384584+0000","last_clean_scrub_stamp":"2023-12-16T21:42:40.384584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:04:04.955333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-16T21:42:42.471919+0000","last_change":"2023-12-16T21:42:40.439054+0000","last_active":"2023-12-16T21:42:42.471919+0000","last_peered":"2023-12-16T21:42:42.471919+0000","last_clean":"2023-12-16T21:42:42.471919+0000","last_became_active":"2023-12-16T21:42:40.438547+0000","last_became_peered":"2023-12-16T21:42:40.438547+0000","last_unstale":"2023-12-16T21:42:42.471919+0000","last_undegraded":"2023-12-16T21:42:42.471919+0000","last_fullsized":"2023-12-16T21:42:42.471919+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:39.372643+0000","last_clean_scrub_stamp":"2023-12-16T21:42:39.372643+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:46:49.119184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.022}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.286}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91900000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1619999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3049999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3400000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0389999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.22}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.874}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.077}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31408,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340432,"statfs":{"total":96636764160,"available":96604602368,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84399999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0209999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6619999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3839999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1519999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.748}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:42:47.442 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-16T21:42:47.442 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-16T21:42:47.442 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-16T21:42:47.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-16T21:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 Processor -- start 2023-12-16T21:42:47.599 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 -- start start 2023-12-16T21:42:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f208006c920 0x7f20800ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f20800af2e0 0x7f20800ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20800b4430 0x7f2080147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20800afc20 con 0x7f20800b4430 2023-12-16T21:42:47.600 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20800af770 con 0x7f20800af2e0 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2087cb5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20800aeea0 con 0x7f208006c920 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20800b4430 0x7f2080147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f20800b4430 0x7f2080147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45266/0 (socket says 172.21.15.46:45266) 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 -- 172.21.15.46:0/251430731 learned_addr learned my addr 172.21.15.46:0/251430731 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2085a51640 1 --2- 172.21.15.46:0/251430731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f208006c920 0x7f20800ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2085250640 1 --2- 172.21.15.46:0/251430731 >> v2:172.21.15.161:3300/0 conn(0x7f20800af2e0 0x7f20800ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 -- 172.21.15.46:0/251430731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f208006c920 msgr2=0x7f20800ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 --2- 172.21.15.46:0/251430731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f208006c920 0x7f20800ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 -- 172.21.15.46:0/251430731 >> v2:172.21.15.161:3300/0 conn(0x7f20800af2e0 msgr2=0x7f20800ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 --2- 172.21.15.46:0/251430731 >> v2:172.21.15.161:3300/0 conn(0x7f20800af2e0 0x7f20800ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2086252640 1 -- 172.21.15.46:0/251430731 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208006cd00 con 0x7f20800b4430 2023-12-16T21:42:47.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.591+0000 7f2085a51640 1 --2- 172.21.15.46:0/251430731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f208006c920 0x7f20800ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:47.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2086252640 1 --2- 172.21.15.46:0/251430731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20800b4430 0x7f2080147080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f208006cef0 tx=0x7f207801c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9a387f997e456e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2084a4f640 1 -- 172.21.15.46:0/251430731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20780a5750 con 0x7f20800b4430 2023-12-16T21:42:47.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2084a4f640 1 -- 172.21.15.46:0/251430731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20780a58b0 con 0x7f20800b4430 2023-12-16T21:42:47.603 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2084a4f640 1 -- 172.21.15.46:0/251430731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20780a5c40 con 0x7f20800b4430 2023-12-16T21:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 -- 172.21.15.46:0/251430731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20800b4430 msgr2=0x7f2080147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 --2- 172.21.15.46:0/251430731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20800b4430 0x7f2080147080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f208006cef0 tx=0x7f207801c8b0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 -- 172.21.15.46:0/251430731 shutdown_connections 2023-12-16T21:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 --2- 172.21.15.46:0/251430731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20800b4430 0x7f2080147080 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 --2- 172.21.15.46:0/251430731 >> v2:172.21.15.161:3300/0 conn(0x7f20800af2e0 0x7f20800ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 --2- 172.21.15.46:0/251430731 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f208006c920 0x7f20800ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.604 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 -- 172.21.15.46:0/251430731 >> 172.21.15.46:0/251430731 conn(0x7f20800b48e0 msgr2=0x7f20800b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 -- 172.21.15.46:0/251430731 shutdown_connections 2023-12-16T21:42:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.595+0000 7f2087cb5640 1 -- 172.21.15.46:0/251430731 wait complete. 2023-12-16T21:42:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 Processor -- start 2023-12-16T21:42:47.605 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 -- start start 2023-12-16T21:42:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006c920 0x7f20800b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20800b4430 0x7f20801431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080143710 0x7f20801df640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2080144840 con 0x7f208006c920 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20801462b0 con 0x7f2080143710 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2080146880 con 0x7f20800b4430 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006c920 0x7f20800b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006c920 0x7f20800b76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45276/0 (socket says 172.21.15.46:45276) 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 -- 172.21.15.46:0/2090265669 learned_addr learned my addr 172.21.15.46:0/2090265669 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085250640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20800b4430 0x7f20801431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 -- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20800b4430 msgr2=0x7f20801431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20800b4430 0x7f20801431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2086252640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080143710 0x7f20801df640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 -- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080143710 msgr2=0x7f20801df640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080143710 0x7f20801df640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 -- 172.21.15.46:0/2090265669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2078009b30 con 0x7f208006c920 2023-12-16T21:42:47.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2085a51640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006c920 0x7f20800b76b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f207000e840 tx=0x7f207000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f20767fc640 1 -- 172.21.15.46:0/2090265669 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070004030 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f20767fc640 1 -- 172.21.15.46:0/2090265669 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2070004190 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2080072330 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f20767fc640 1 -- 172.21.15.46:0/2090265669 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2070004520 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20801464a0 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.599+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20800679e0 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.603+0000 7f20767fc640 1 -- 172.21.15.46:0/2090265669 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f207000c6d0 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.603+0000 7f20767fc640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2064066470 0x7f2064068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.603+0000 7f20767fc640 1 -- 172.21.15.46:0/2090265669 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2070083990 con 0x7f208006c920 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.603+0000 7f2085250640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2064066470 0x7f2064068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.603+0000 7f2085250640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2064066470 0x7f2064068930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2080145ae0 tx=0x7f207c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.607+0000 7f20767fc640 1 -- 172.21.15.46:0/2090265669 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2070050ea0 con 0x7f208006c920 2023-12-16T21:42:47.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f20800688f0 con 0x7f208006c920 2023-12-16T21:42:47.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f20767fc640 1 -- 172.21.15.46:0/2090265669 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f20700544c0 con 0x7f208006c920 2023-12-16T21:42:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:47.794 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-16T21:42:47.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2064066470 msgr2=0x7f2064068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2064066470 0x7f2064068930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2080145ae0 tx=0x7f207c008040 comp rx=0 tx=0).stop 2023-12-16T21:42:47.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006c920 msgr2=0x7f20800b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006c920 0x7f20800b76b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f207000e840 tx=0x7f207000ed00 comp rx=0 tx=0).stop 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 shutdown_connections 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2064066470 0x7f2064068930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2080143710 0x7f20801df640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f20800b4430 0x7f20801431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 --2- 172.21.15.46:0/2090265669 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f208006c920 0x7f20800b76b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 >> 172.21.15.46:0/2090265669 conn(0x7f20800b48e0 msgr2=0x7f208013fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 shutdown_connections 2023-12-16T21:42:47.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.787+0000 7f2087cb5640 1 -- 172.21.15.46:0/2090265669 wait complete. 2023-12-16T21:42:47.811 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-16T21:42:47.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62f351640 1 Processor -- start 2023-12-16T21:42:47.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62f351640 1 -- start start 2023-12-16T21:42:47.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62f351640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe62806c920 0x7fe6280afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62f351640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 0x7fe62813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62f351640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6280af2e0 con 0x7fe6280b3fd0 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62f351640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6280b1f50 con 0x7fe62806c920 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62f351640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62806cd00 con 0x7fe62809dd50 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62c8ec640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe6280b3fd0 0x7fe62813e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:43108/0 (socket says 172.21.15.46:43108) 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62c8ec640 1 -- 172.21.15.46:0/2972513778 learned_addr learned my addr 172.21.15.46:0/2972513778 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d8ee640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 0x7fe62813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d0ed640 1 --2- 172.21.15.46:0/2972513778 >> v2:172.21.15.161:3300/0 conn(0x7fe62806c920 0x7fe6280afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384572861 0 0) 0x7fe6280af2e0 con 0x7fe6280b3fd0 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe614003530 con 0x7fe6280b3fd0 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994044266 0 0) 0x7fe614003530 con 0x7fe6280b3fd0 2023-12-16T21:42:47.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 msgr2=0x7fe62813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 0x7fe62813ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 >> v2:172.21.15.161:3300/0 conn(0x7fe62806c920 msgr2=0x7fe6280afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 --2- 172.21.15.46:0/2972513778 >> v2:172.21.15.161:3300/0 conn(0x7fe62806c920 0x7fe6280afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280af530 con 0x7fe6280b3fd0 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fe618001680 con 0x7fe6280b3fd0 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 >> v1:172.21.15.46:6789/0 conn(0x7fe6280b3fd0 legacy=0x7fe62813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 0x7fe614003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614003f00 0x7fe6140047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe614004d10 0x7fe614005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6280b1f50 con 0x7fe614004d10 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62806cd00 con 0x7fe614003f00 2023-12-16T21:42:47.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61400b9e0 con 0x7fe62809dd50 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d8ee640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe614004d10 0x7fe614005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d0ed640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 0x7fe614003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62c8ec640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614003f00 0x7fe6140047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d8ee640 1 -- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 msgr2=0x7fe614003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d8ee640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 0x7fe614003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d8ee640 1 -- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614003f00 msgr2=0x7fe6140047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d8ee640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614003f00 0x7fe6140047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d8ee640 1 -- 172.21.15.46:0/2972513778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe614003530 con 0x7fe614004d10 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62d0ed640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62809dd50 0x7fe614003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:47.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.915+0000 7fe62c8ec640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614003f00 0x7fe6140047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62d8ee640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe614004d10 0x7fe614005190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe620009f90 tx=0x7fe62001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2a44510f2fe0d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6200a5bf0 con 0x7fe614004d10 2023-12-16T21:42:47.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62000f040 con 0x7fe614004d10 2023-12-16T21:42:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe61ffff640 1 -- 172.21.15.46:0/2972513778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6200ad440 con 0x7fe614004d10 2023-12-16T21:42:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- 172.21.15.46:0/2972513778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe614004d10 msgr2=0x7fe614005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 --2- 172.21.15.46:0/2972513778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe614004d10 0x7fe614005190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe620009f90 tx=0x7fe62001cbe0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62d8ee640 1 -- 172.21.15.46:0/2972513778 reap_dead start 2023-12-16T21:42:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- 172.21.15.46:0/2972513778 shutdown_connections 2023-12-16T21:42:47.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- 172.21.15.46:0/2972513778 >> 172.21.15.46:0/2972513778 conn(0x7fe6280b48e0 msgr2=0x7fe6280b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- 172.21.15.46:0/2972513778 shutdown_connections 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- 172.21.15.46:0/2972513778 wait complete. 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 Processor -- start 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- start start 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614004d10 0x7fe6281448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6281dd030 0x7fe62813ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62813f4c0 0x7fe6281d3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6281ddb80 con 0x7fe6281dd030 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62806f9e0 con 0x7fe614004d10 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62806fec0 con 0x7fe62813f4c0 2023-12-16T21:42:47.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6281dd030 0x7fe62813ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6281dd030 0x7fe62813ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45286/0 (socket says 172.21.15.46:45286) 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 -- 172.21.15.46:0/4101742940 learned_addr learned my addr 172.21.15.46:0/4101742940 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62d0ed640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614004d10 0x7fe6281448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62d8ee640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62813f4c0 0x7fe6281d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 -- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62813f4c0 msgr2=0x7fe6281d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62813f4c0 0x7fe6281d3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 -- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614004d10 msgr2=0x7fe6281448a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614004d10 0x7fe6281448a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 -- 172.21.15.46:0/4101742940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6280b4750 con 0x7fe6281dd030 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62c8ec640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6281dd030 0x7fe62813ef80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe618011810 tx=0x7fe618011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe61dffb640 1 -- 172.21.15.46:0/4101742940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61800fb60 con 0x7fe6281dd030 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe61dffb640 1 -- 172.21.15.46:0/4101742940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61800fce0 con 0x7fe6281dd030 2023-12-16T21:42:47.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6281dd4c0 con 0x7fe6281dd030 2023-12-16T21:42:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe61dffb640 1 -- 172.21.15.46:0/4101742940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe618015650 con 0x7fe6281dd030 2023-12-16T21:42:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.919+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62806fbe0 con 0x7fe6281dd030 2023-12-16T21:42:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.923+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6280679e0 con 0x7fe6281dd030 2023-12-16T21:42:47.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.923+0000 7fe61dffb640 1 -- 172.21.15.46:0/4101742940 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fe6180157b0 con 0x7fe6281dd030 2023-12-16T21:42:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.923+0000 7fe61dffb640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe60c066470 0x7fe60c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.923+0000 7fe61dffb640 1 -- 172.21.15.46:0/4101742940 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6180870c0 con 0x7fe6281dd030 2023-12-16T21:42:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.923+0000 7fe62d0ed640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe60c066470 0x7fe60c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.923+0000 7fe62d0ed640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe60c066470 0x7fe60c068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe624009b30 tx=0x7fe624008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:47.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:47.927+0000 7fe61dffb640 1 -- 172.21.15.46:0/4101742940 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fe618054540 con 0x7fe6281dd030 2023-12-16T21:42:48.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.079+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6281ddcc0 con 0x7fe6281dd030 2023-12-16T21:42:48.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.079+0000 7fe61dffb640 1 -- 172.21.15.46:0/4101742940 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fe6281ddcc0 con 0x7fe6281dd030 2023-12-16T21:42:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:48.088 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:48.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe60c066470 msgr2=0x7fe60c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe60c066470 0x7fe60c068930 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe624009b30 tx=0x7fe624008040 comp rx=0 tx=0).stop 2023-12-16T21:42:48.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6281dd030 msgr2=0x7fe62813ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6281dd030 0x7fe62813ef80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe618011810 tx=0x7fe618011cd0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 shutdown_connections 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe60c066470 0x7fe60c068930 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe62813f4c0 0x7fe6281d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe6281dd030 0x7fe62813ef80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 --2- 172.21.15.46:0/4101742940 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe614004d10 0x7fe6281448a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 >> 172.21.15.46:0/4101742940 conn(0x7fe6280b48e0 msgr2=0x7fe62806be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 shutdown_connections 2023-12-16T21:42:48.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.083+0000 7fe62f351640 1 -- 172.21.15.46:0/4101742940 wait complete. 2023-12-16T21:42:48.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:48.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 Processor -- start 2023-12-16T21:42:48.227 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 -- start start 2023-12-16T21:42:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b1eb0 0x7f78e00aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f78e00af300 0x7f78e0144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.228 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78e00b0470 0x7f78e0144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e006cd70 con 0x7f78e00b0470 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e00b0ad0 con 0x7f78e00af300 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78e590d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00b1730 con 0x7f78e00b1eb0 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78e00b0470 0x7f78e0144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f78e00b0470 0x7f78e0144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45290/0 (socket says 172.21.15.46:45290) 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 -- 172.21.15.46:0/827028616 learned_addr learned my addr 172.21.15.46:0/827028616 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78deffd640 1 --2- 172.21.15.46:0/827028616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b1eb0 0x7f78e00aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78de7fc640 1 --2- 172.21.15.46:0/827028616 >> v2:172.21.15.161:3300/0 conn(0x7f78e00af300 0x7f78e0144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 -- 172.21.15.46:0/827028616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b1eb0 msgr2=0x7f78e00aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 --2- 172.21.15.46:0/827028616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b1eb0 0x7f78e00aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 -- 172.21.15.46:0/827028616 >> v2:172.21.15.161:3300/0 conn(0x7f78e00af300 msgr2=0x7f78e0144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:48.229 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 --2- 172.21.15.46:0/827028616 >> v2:172.21.15.161:3300/0 conn(0x7f78e00af300 0x7f78e0144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78df7fe640 1 -- 172.21.15.46:0/827028616 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e006c7f0 con 0x7f78e00b0470 2023-12-16T21:42:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.219+0000 7f78de7fc640 1 --2- 172.21.15.46:0/827028616 >> v2:172.21.15.161:3300/0 conn(0x7f78e00af300 0x7f78e0144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78df7fe640 1 --2- 172.21.15.46:0/827028616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b0470 0x7f78e0144ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f78e00af170 tx=0x7f78d001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c0c53c7aa8a78e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78ddffb640 1 -- 172.21.15.46:0/827028616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00ac600 con 0x7f78e00b0470 2023-12-16T21:42:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78ddffb640 1 -- 172.21.15.46:0/827028616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d00a8050 con 0x7f78e00b0470 2023-12-16T21:42:48.230 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78ddffb640 1 -- 172.21.15.46:0/827028616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00ac980 con 0x7f78e00b0470 2023-12-16T21:42:48.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 -- 172.21.15.46:0/827028616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b0470 msgr2=0x7f78e0144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 --2- 172.21.15.46:0/827028616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b0470 0x7f78e0144ff0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f78e00af170 tx=0x7f78d001c3e0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 -- 172.21.15.46:0/827028616 shutdown_connections 2023-12-16T21:42:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 --2- 172.21.15.46:0/827028616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b0470 0x7f78e0144ff0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 --2- 172.21.15.46:0/827028616 >> v2:172.21.15.161:3300/0 conn(0x7f78e00af300 0x7f78e0144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 --2- 172.21.15.46:0/827028616 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b1eb0 0x7f78e00aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 -- 172.21.15.46:0/827028616 >> 172.21.15.46:0/827028616 conn(0x7f78e00b4900 msgr2=0x7f78e00b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 -- 172.21.15.46:0/827028616 shutdown_connections 2023-12-16T21:42:48.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 -- 172.21.15.46:0/827028616 wait complete. 2023-12-16T21:42:48.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 Processor -- start 2023-12-16T21:42:48.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.223+0000 7f78e590d640 1 -- start start 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78e00af300 0x7f78e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b0470 0x7f78e0146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b1eb0 0x7f78e013ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e013a8d0 con 0x7f78e00b1eb0 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e00709b0 con 0x7f78e00af300 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e0070c90 con 0x7f78e00b0470 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b1eb0 0x7f78e013ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b1eb0 0x7f78e013ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45306/0 (socket says 172.21.15.46:45306) 2023-12-16T21:42:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 -- 172.21.15.46:0/3025435142 learned_addr learned my addr 172.21.15.46:0/3025435142 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78de7fc640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b0470 0x7f78e0146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78deffd640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78e00af300 0x7f78e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 -- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b0470 msgr2=0x7f78e0146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b0470 0x7f78e0146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 -- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78e00af300 msgr2=0x7f78e006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78e00af300 0x7f78e006edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 -- 172.21.15.46:0/3025435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00094d0 con 0x7f78e00b1eb0 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78deffd640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78e00af300 0x7f78e006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78df7fe640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b1eb0 0x7f78e013ad60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f78d001aa90 tx=0x7f78d001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78c7fff640 1 -- 172.21.15.46:0/3025435142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00b7860 con 0x7f78e00b1eb0 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78c7fff640 1 -- 172.21.15.46:0/3025435142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d00b79c0 con 0x7f78e00b1eb0 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e00712e0 con 0x7f78e00b1eb0 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78c7fff640 1 -- 172.21.15.46:0/3025435142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d00b7d70 con 0x7f78e00b1eb0 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e0147740 con 0x7f78e00b1eb0 2023-12-16T21:42:48.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e0071790 con 0x7f78e00b1eb0 2023-12-16T21:42:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78c7fff640 1 -- 172.21.15.46:0/3025435142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f78d00b3070 con 0x7f78e00b1eb0 2023-12-16T21:42:48.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78c7fff640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f78cc066470 0x7f78cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78c7fff640 1 -- 172.21.15.46:0/3025435142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78d011e720 con 0x7f78e00b1eb0 2023-12-16T21:42:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.227+0000 7f78deffd640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f78cc066470 0x7f78cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.231+0000 7f78deffd640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f78cc066470 0x7f78cc068930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78e006ca10 tx=0x7f78d800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.231+0000 7f78c7fff640 1 -- 172.21.15.46:0/3025435142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f78d00ebb90 con 0x7f78e00b1eb0 2023-12-16T21:42:48.392 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.383+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f78e0071120 con 0x7f78e00b1eb0 2023-12-16T21:42:48.393 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78c7fff640 1 -- 172.21.15.46:0/3025435142 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f78d00ace60 con 0x7f78e00b1eb0 2023-12-16T21:42:48.393 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f78cc066470 msgr2=0x7f78cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f78cc066470 0x7f78cc068930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f78e006ca10 tx=0x7f78d800b040 comp rx=0 tx=0).stop 2023-12-16T21:42:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b1eb0 msgr2=0x7f78e013ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b1eb0 0x7f78e013ad60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f78d001aa90 tx=0x7f78d001ac20 comp rx=0 tx=0).stop 2023-12-16T21:42:48.395 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 shutdown_connections 2023-12-16T21:42:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f78cc066470 0x7f78cc068930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f78e00b1eb0 0x7f78e013ad60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f78e00b0470 0x7f78e0146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 --2- 172.21.15.46:0/3025435142 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f78e00af300 0x7f78e006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 >> 172.21.15.46:0/3025435142 conn(0x7f78e00b4900 msgr2=0x7f78e0143620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 shutdown_connections 2023-12-16T21:42:48.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.387+0000 7f78e590d640 1 -- 172.21.15.46:0/3025435142 wait complete. 2023-12-16T21:42:48.411 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:42:48.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8e63f640 1 Processor -- start 2023-12-16T21:42:48.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8e63f640 1 -- start start 2023-12-16T21:42:48.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8e63f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d8806cdc0 0x7f7d8806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8e63f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7d880b5330 0x7f7d881405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8e63f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d880afb30 con 0x7f7d880b40b0 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8e63f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d880704b0 con 0x7f7d880b5330 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8e63f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806f8c0 con 0x7f7d8806cdc0 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d877fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7d880b40b0 0x7f7d88071590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:41998/0 (socket says 172.21.15.46:41998) 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d877fe640 1 -- 172.21.15.46:0/3550611433 learned_addr learned my addr 172.21.15.46:0/3550611433 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541857092 0 0) 0x7f7d880afb30 con 0x7f7d880b40b0 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d87fff640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d8806cdc0 0x7f7d8806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d8cbdc640 1 --2- 172.21.15.46:0/3550611433 >> v2:172.21.15.161:3300/0 conn(0x7f7d880b5330 0x7f7d881405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d70003530 con 0x7f7d880b40b0 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 331041040 0 0) 0x7f7d70003530 con 0x7f7d880b40b0 2023-12-16T21:42:48.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d8806cdc0 msgr2=0x7f7d8806fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d8806cdc0 0x7f7d8806fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 >> v2:172.21.15.161:3300/0 conn(0x7f7d880b5330 msgr2=0x7f7d881405e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 --2- 172.21.15.46:0/3550611433 >> v2:172.21.15.161:3300/0 conn(0x7f7d880b5330 0x7f7d881405e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8806d1a0 con 0x7f7d880b40b0 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f7d7c001680 con 0x7f7d880b40b0 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 >> v1:172.21.15.46:6789/0 conn(0x7f7d880b40b0 legacy=0x7f7d88071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70003980 0x7f7d70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d700047d0 0x7f7d70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d8806cdc0 0x7f7d70005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880704b0 con 0x7f7d70003980 2023-12-16T21:42:48.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806f8c0 con 0x7f7d700047d0 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7000b9e0 con 0x7f7d8806cdc0 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d87fff640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70003980 0x7f7d70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d877fe640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d700047d0 0x7f7d70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d87fff640 1 -- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d8806cdc0 msgr2=0x7f7d70005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d87fff640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d8806cdc0 0x7f7d70005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d87fff640 1 -- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d700047d0 msgr2=0x7f7d70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d87fff640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d700047d0 0x7f7d70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d87fff640 1 -- 172.21.15.46:0/3550611433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d70003530 con 0x7f7d70003980 2023-12-16T21:42:48.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.531+0000 7f7d877fe640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d700047d0 0x7f7d70004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d87fff640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70003980 0x7f7d70003da0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e170 tx=0x7f7d7801cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca043a870f2f5fbb server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780a5a70 con 0x7f7d70003980 2023-12-16T21:42:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7800f040 con 0x7f7d70003980 2023-12-16T21:42:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d86ffd640 1 -- 172.21.15.46:0/3550611433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780a5e10 con 0x7f7d70003980 2023-12-16T21:42:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- 172.21.15.46:0/3550611433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70003980 msgr2=0x7f7d70003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 --2- 172.21.15.46:0/3550611433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d70003980 0x7f7d70003da0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e170 tx=0x7f7d7801cc50 comp rx=0 tx=0).stop 2023-12-16T21:42:48.545 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 -- 172.21.15.46:0/3550611433 reap_dead start 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- 172.21.15.46:0/3550611433 shutdown_connections 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- 172.21.15.46:0/3550611433 >> 172.21.15.46:0/3550611433 conn(0x7f7d880b90b0 msgr2=0x7f7d880b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- 172.21.15.46:0/3550611433 shutdown_connections 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- 172.21.15.46:0/3550611433 wait complete. 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 Processor -- start 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- start start 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d70003980 0x7f7d8813ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d8814ec90 0x7f7d8814f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881e2860 0x7f7d881e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8814e560 con 0x7f7d881e2860 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8814f610 con 0x7f7d8814ec90 2023-12-16T21:42:48.546 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8e63f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d881e35a0 con 0x7f7d70003980 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881e2860 0x7f7d881e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881e2860 0x7f7d881e2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51330/0 (socket says 172.21.15.46:51330) 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 -- 172.21.15.46:0/2052567014 learned_addr learned my addr 172.21.15.46:0/2052567014 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d87fff640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d70003980 0x7f7d8813ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d877fe640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d8814ec90 0x7f7d8814f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 -- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d70003980 msgr2=0x7f7d8813ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d70003980 0x7f7d8813ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 -- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d8814ec90 msgr2=0x7f7d8814f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d8814ec90 0x7f7d8814f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 -- 172.21.15.46:0/2052567014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8814e770 con 0x7f7d881e2860 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d877fe640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d8814ec90 0x7f7d8814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d8cbdc640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881e2860 0x7f7d881e2d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b6870 tx=0x7f7d8000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.535+0000 7f7d84ff9640 1 -- 172.21.15.46:0/2052567014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000c680 con 0x7f7d881e2860 2023-12-16T21:42:48.547 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d84ff9640 1 -- 172.21.15.46:0/2052567014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000c800 con 0x7f7d881e2860 2023-12-16T21:42:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d880b2ab0 con 0x7f7d881e2860 2023-12-16T21:42:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d84ff9640 1 -- 172.21.15.46:0/2052567014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000cbb0 con 0x7f7d881e2860 2023-12-16T21:42:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d88147cb0 con 0x7f7d881e2860 2023-12-16T21:42:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d88067f60 con 0x7f7d881e2860 2023-12-16T21:42:48.548 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d84ff9640 1 -- 172.21.15.46:0/2052567014 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7d8000cd10 con 0x7f7d881e2860 2023-12-16T21:42:48.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d84ff9640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d740664a0 0x7f7d74068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d84ff9640 1 -- 172.21.15.46:0/2052567014 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d80083590 con 0x7f7d881e2860 2023-12-16T21:42:48.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d87fff640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d740664a0 0x7f7d74068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.549 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.539+0000 7f7d87fff640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d740664a0 0x7f7d74068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7d780af000 tx=0x7f7d780a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.550 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.543+0000 7f7d84ff9640 1 -- 172.21.15.46:0/2052567014 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7d80050970 con 0x7f7d881e2860 2023-12-16T21:42:48.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d881e3760 con 0x7f7d881e2860 2023-12-16T21:42:48.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d84ff9640 1 -- 172.21.15.46:0/2052567014 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d80053f90 con 0x7f7d881e2860 2023-12-16T21:42:48.705 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:42:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d740664a0 msgr2=0x7f7d74068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d740664a0 0x7f7d74068960 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7d780af000 tx=0x7f7d780a7040 comp rx=0 tx=0).stop 2023-12-16T21:42:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881e2860 msgr2=0x7f7d881e2d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881e2860 0x7f7d881e2d10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b6870 tx=0x7f7d8000ec00 comp rx=0 tx=0).stop 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 shutdown_connections 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7d740664a0 0x7f7d74068960 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d881e2860 0x7f7d881e2d10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7d8814ec90 0x7f7d8814f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 --2- 172.21.15.46:0/2052567014 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7d70003980 0x7f7d8813ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 >> 172.21.15.46:0/2052567014 conn(0x7f7d880b90b0 msgr2=0x7f7d881506d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 shutdown_connections 2023-12-16T21:42:48.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.699+0000 7f7d8e63f640 1 -- 172.21.15.46:0/2052567014 wait complete. 2023-12-16T21:42:48.723 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:42:48.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff0070c3640 1 Processor -- start 2023-12-16T21:42:48.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff0070c3640 1 -- start start 2023-12-16T21:42:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff0070c3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0000ba920 0x7ff000140580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff0070c3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0000b52e0 0x7ff000140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff0070c3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0000b5d80 con 0x7ff00006c8f0 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff0070c3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0000bad00 con 0x7ff0000b52e0 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff0070c3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b6020 con 0x7ff0000ba920 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7feffffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff0000b52e0 0x7ff000140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff005660640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff00006c8f0 0x7ff00006cd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42012/0 (socket says 172.21.15.46:42012) 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff005660640 1 -- 172.21.15.46:0/3662710601 learned_addr learned my addr 172.21.15.46:0/3662710601 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7ff004e5f640 1 --2- 172.21.15.46:0/3662710601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0000ba920 0x7ff000140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7feffffff640 1 -- 172.21.15.46:0/3662710601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0000ba920 msgr2=0x7ff000140580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7feffffff640 1 --2- 172.21.15.46:0/3662710601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0000ba920 0x7ff000140580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7feffffff640 1 -- 172.21.15.46:0/3662710601 >> v1:172.21.15.46:6789/0 conn(0x7ff00006c8f0 legacy=0x7ff00006cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7feffffff640 1 -- 172.21.15.46:0/3662710601 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000b6300 con 0x7ff0000b52e0 2023-12-16T21:42:48.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7fefff7fe640 1 -- 172.21.15.46:0/3662710601 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151834829 0 0) 0x7ff0000b5d80 con 0x7ff00006c8f0 2023-12-16T21:42:48.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7feffffff640 1 --2- 172.21.15.46:0/3662710601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff000140ac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feff40097c0 tx=0x7feff401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2201de3cccea92 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.831+0000 7fefff7fe640 1 -- 172.21.15.46:0/3662710601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40ab600 con 0x7ff0000b52e0 2023-12-16T21:42:48.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7fefff7fe640 1 -- 172.21.15.46:0/3662710601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff40a7050 con 0x7ff0000b52e0 2023-12-16T21:42:48.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7fefff7fe640 1 -- 172.21.15.46:0/3662710601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40ab980 con 0x7ff0000b52e0 2023-12-16T21:42:48.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- 172.21.15.46:0/3662710601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 msgr2=0x7ff000140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3662710601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff000140ac0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7feff40097c0 tx=0x7feff401c360 comp rx=0 tx=0).stop 2023-12-16T21:42:48.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- 172.21.15.46:0/3662710601 shutdown_connections 2023-12-16T21:42:48.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3662710601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff000140ac0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3662710601 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0000ba920 0x7ff000140580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- 172.21.15.46:0/3662710601 >> 172.21.15.46:0/3662710601 conn(0x7ff00006ab80 msgr2=0x7ff0000b9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:48.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- 172.21.15.46:0/3662710601 shutdown_connections 2023-12-16T21:42:48.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- 172.21.15.46:0/3662710601 wait complete. 2023-12-16T21:42:48.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 Processor -- start 2023-12-16T21:42:48.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- start start 2023-12-16T21:42:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff00006c8f0 0x7ff0001e2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff00013ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0000ba920 0x7ff00013b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001e2b20 con 0x7ff0000b52e0 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff000146dd0 con 0x7ff0000ba920 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000b1530 con 0x7ff00006c8f0 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff00013ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff00013ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51342/0 (socket says 172.21.15.46:51342) 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 -- 172.21.15.46:0/3192302171 learned_addr learned my addr 172.21.15.46:0/3192302171 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff004e5f640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff00006c8f0 0x7ff0001e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff005660640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0000ba920 0x7ff00013b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 -- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff00006c8f0 msgr2=0x7ff0001e2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff00006c8f0 0x7ff0001e2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 -- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0000ba920 msgr2=0x7ff00013b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0000ba920 0x7ff00013b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:48.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 -- 172.21.15.46:0/3192302171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4009420 con 0x7ff0000b52e0 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff005660640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0000ba920 0x7ff00013b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffffff640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff00013ab10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feff4019f90 tx=0x7feff401aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffd7fa640 1 -- 172.21.15.46:0/3192302171 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40b7650 con 0x7ff0000b52e0 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffd7fa640 1 -- 172.21.15.46:0/3192302171 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff40abe90 con 0x7ff0000b52e0 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0001e2740 con 0x7ff0000b52e0 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7feffd7fa640 1 -- 172.21.15.46:0/3192302171 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40b7a10 con 0x7ff0000b52e0 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.835+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0001443a0 con 0x7ff0000b52e0 2023-12-16T21:42:48.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.839+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00006bb50 con 0x7ff0000b52e0 2023-12-16T21:42:48.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.839+0000 7feffd7fa640 1 -- 172.21.15.46:0/3192302171 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7feff40b7b70 con 0x7ff0000b52e0 2023-12-16T21:42:48.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.839+0000 7feffd7fa640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fefec0664a0 0x7fefec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:48.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.839+0000 7feffd7fa640 1 -- 172.21.15.46:0/3192302171 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feff411e830 con 0x7ff0000b52e0 2023-12-16T21:42:48.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.839+0000 7ff004e5f640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fefec0664a0 0x7fefec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:48.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.839+0000 7ff004e5f640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fefec0664a0 0x7fefec068960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feff000b9d0 tx=0x7feff0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:48.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.843+0000 7feffd7fa640 1 -- 172.21.15.46:0/3192302171 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7feff40ebdd0 con 0x7ff0000b52e0 2023-12-16T21:42:48.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.991+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff000146c20 con 0x7ff0000b52e0 2023-12-16T21:42:49.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.991+0000 7feffd7fa640 1 -- 172.21.15.46:0/3192302171 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7feff40b00a0 con 0x7ff0000b52e0 2023-12-16T21:42:49.000 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:42:49.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fefec0664a0 msgr2=0x7fefec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fefec0664a0 0x7fefec068960 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feff000b9d0 tx=0x7feff0008040 comp rx=0 tx=0).stop 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 msgr2=0x7ff00013ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff00013ab10 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7feff4019f90 tx=0x7feff401aba0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 shutdown_connections 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fefec0664a0 0x7fefec068960 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff0000ba920 0x7ff00013b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff0000b52e0 0x7ff00013ab10 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 --2- 172.21.15.46:0/3192302171 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff00006c8f0 0x7ff0001e2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 >> 172.21.15.46:0/3192302171 conn(0x7ff00006ab80 msgr2=0x7ff0001390b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 shutdown_connections 2023-12-16T21:42:49.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:48.995+0000 7ff0070c3640 1 -- 172.21.15.46:0/3192302171 wait complete. 2023-12-16T21:42:49.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:42:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae6653640 1 Processor -- start 2023-12-16T21:42:49.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae6653640 1 -- start start 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae6653640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae0071cd0 0x7fcae00b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae6653640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcae00b2990 0x7fcae00b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae6653640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae00720b0 con 0x7fcae0144740 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae6653640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae013eca0 con 0x7fcae00b2990 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae6653640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00b4c90 con 0x7fcae0071cd0 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fcae0144740 0x7fcae0144b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42022/0 (socket says 172.21.15.46:42022) 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 -- 172.21.15.46:0/3149061939 learned_addr learned my addr 172.21.15.46:0/3149061939 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadf7fe640 1 --2- 172.21.15.46:0/3149061939 >> v2:172.21.15.161:3300/0 conn(0x7fcae00b2990 0x7fcae00b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadffff640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae0071cd0 0x7fcae00b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519594778 0 0) 0x7fcae00720b0 con 0x7fcae0144740 2023-12-16T21:42:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac8003530 con 0x7fcae0144740 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 475869299 0 0) 0x7fcac8003530 con 0x7fcae0144740 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae0071cd0 msgr2=0x7fcae00b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae0071cd0 0x7fcae00b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 >> v2:172.21.15.161:3300/0 conn(0x7fcae00b2990 msgr2=0x7fcae00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 --2- 172.21.15.46:0/3149061939 >> v2:172.21.15.161:3300/0 conn(0x7fcae00b2990 0x7fcae00b2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00b4f70 con 0x7fcae0144740 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fcad8001a00 con 0x7fcae0144740 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadf7fe640 1 --2- 172.21.15.46:0/3149061939 >> v2:172.21.15.161:3300/0 conn(0x7fcae00b2990 0x7fcae00b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 >> v1:172.21.15.46:6789/0 conn(0x7fcae0144740 legacy=0x7fcae0144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac8003980 0x7fcac8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcac80047d0 0x7fcac8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8005190 0x7fcac800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae013eca0 con 0x7fcac8005190 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00b4c90 con 0x7fcac8003980 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac800bf20 con 0x7fcac80047d0 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadffff640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae0071cd0 0x7fcae00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8005190 0x7fcac800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcadf7fe640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcac80047d0 0x7fcac8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 -- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcac80047d0 msgr2=0x7fcac8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcac80047d0 0x7fcac8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 -- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac8003980 msgr2=0x7fcac8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcac8003980 0x7fcac8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.135+0000 7fcae4bf0640 1 -- 172.21.15.46:0/3149061939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac8003530 con 0x7fcac8005190 2023-12-16T21:42:49.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae4bf0640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8005190 0x7fcac800b990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcad800a860 tx=0x7fcad801faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=378688e364f0b884 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad80aa6d0 con 0x7fcac8005190 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad80aa830 con 0x7fcac8005190 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadeffd640 1 -- 172.21.15.46:0/3149061939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad80aac00 con 0x7fcac8005190 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- 172.21.15.46:0/3149061939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8005190 msgr2=0x7fcac800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 --2- 172.21.15.46:0/3149061939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcac8005190 0x7fcac800b990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcad800a860 tx=0x7fcad801faa0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae4bf0640 1 -- 172.21.15.46:0/3149061939 reap_dead start 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- 172.21.15.46:0/3149061939 shutdown_connections 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- 172.21.15.46:0/3149061939 >> 172.21.15.46:0/3149061939 conn(0x7fcae00b5850 msgr2=0x7fcae00b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- 172.21.15.46:0/3149061939 shutdown_connections 2023-12-16T21:42:49.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- 172.21.15.46:0/3149061939 wait complete. 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 Processor -- start 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- start start 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcae01dd260 0x7fcae01ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcae01ddcb0 0x7fcae0139ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae013a000 0x7fcae01d9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01d1400 con 0x7fcae01dd260 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01ccf70 con 0x7fcae01ddcb0 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae6653640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01cbf50 con 0x7fcae013a000 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcae01dd260 0x7fcae01ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcae01dd260 0x7fcae01ca5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51362/0 (socket says 172.21.15.46:51362) 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 -- 172.21.15.46:0/1295333504 learned_addr learned my addr 172.21.15.46:0/1295333504 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadf7fe640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcae01ddcb0 0x7fcae0139ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae4bf0640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae013a000 0x7fcae01d9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 -- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae013a000 msgr2=0x7fcae01d9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae013a000 0x7fcae01d9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 -- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcae01ddcb0 msgr2=0x7fcae0139ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcae01ddcb0 0x7fcae0139ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 -- 172.21.15.46:0/1295333504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae01cab00 con 0x7fcae01dd260 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcae4bf0640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae013a000 0x7fcae01d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.139+0000 7fcadffff640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcae01dd260 0x7fcae01ca5c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcad40047a0 tx=0x7fcad400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadcff9640 1 -- 172.21.15.46:0/1295333504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4004030 con 0x7fcae01dd260 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadcff9640 1 -- 172.21.15.46:0/1295333504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4004190 con 0x7fcae01dd260 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae01cd4a0 con 0x7fcae01dd260 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadcff9640 1 -- 172.21.15.46:0/1295333504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad400c830 con 0x7fcae01dd260 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae01cd1c0 con 0x7fcae01dd260 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae01cbda0 con 0x7fcae01dd260 2023-12-16T21:42:49.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadcff9640 1 -- 172.21.15.46:0/1295333504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fcad400c990 con 0x7fcae01dd260 2023-12-16T21:42:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadcff9640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcacc066470 0x7fcacc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadcff9640 1 -- 172.21.15.46:0/1295333504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcad4083d00 con 0x7fcae01dd260 2023-12-16T21:42:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadf7fe640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcacc066470 0x7fcacc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.143+0000 7fcadf7fe640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcacc066470 0x7fcacc068930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcae013a8d0 tx=0x7fcad0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.147+0000 7fcadcff9640 1 -- 172.21.15.46:0/1295333504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fcad4051210 con 0x7fcae01dd260 2023-12-16T21:42:49.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.327+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcae00b6670 con 0x7fcae01dd260 2023-12-16T21:42:49.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.327+0000 7fcadcff9640 1 -- 172.21.15.46:0/1295333504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7fcad4020d30 con 0x7fcae01dd260 2023-12-16T21:42:49.335 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:49.335 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:42:49.335 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:42:49.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcacc066470 msgr2=0x7fcacc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcacc066470 0x7fcacc068930 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fcae013a8d0 tx=0x7fcad0008040 comp rx=0 tx=0).stop 2023-12-16T21:42:49.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcae01dd260 msgr2=0x7fcae01ca5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcae01dd260 0x7fcae01ca5c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcad40047a0 tx=0x7fcad400ecd0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 shutdown_connections 2023-12-16T21:42:49.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcacc066470 0x7fcacc068930 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcae013a000 0x7fcae01d9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcae01ddcb0 0x7fcae0139ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 --2- 172.21.15.46:0/1295333504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcae01dd260 0x7fcae01ca5c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 >> 172.21.15.46:0/1295333504 conn(0x7fcae00b5850 msgr2=0x7fcae006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:49.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 shutdown_connections 2023-12-16T21:42:49.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.331+0000 7fcae6653640 1 -- 172.21.15.46:0/1295333504 wait complete. 2023-12-16T21:42:49.353 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:49.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.467+0000 7f48db919640 1 Processor -- start 2023-12-16T21:42:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48db919640 1 -- start start 2023-12-16T21:42:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48db919640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48d414a390 0x7f48d40b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48db919640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d413ff50 0x7f48d4140390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48db919640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48d40bcbe0 con 0x7f48d40bc800 2023-12-16T21:42:49.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48db919640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48d40b38f0 con 0x7f48d414a390 2023-12-16T21:42:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48db919640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d40b3bd0 con 0x7f48d413ff50 2023-12-16T21:42:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d8eb4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48d414a390 0x7f48d40b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d9eb6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d413ff50 0x7f48d4140390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d96b5640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f48d40bc800 0x7f48d4149c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:45710/0 (socket says 172.21.15.46:45710) 2023-12-16T21:42:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d96b5640 1 -- 172.21.15.46:0/4014449822 learned_addr learned my addr 172.21.15.46:0/4014449822 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d8eb4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48d414a390 0x7f48d40b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:53630/0 (socket says 172.21.15.46:53630) 2023-12-16T21:42:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d8eb4640 1 -- 172.21.15.46:0/4014449822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d413ff50 msgr2=0x7f48d4140390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d8eb4640 1 --2- 172.21.15.46:0/4014449822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d413ff50 0x7f48d4140390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d8eb4640 1 -- 172.21.15.46:0/4014449822 >> v1:172.21.15.161:6789/0 conn(0x7f48d40bc800 legacy=0x7f48d4149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.471+0000 7f48d8eb4640 1 -- 172.21.15.46:0/4014449822 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d40b3eb0 con 0x7f48d414a390 2023-12-16T21:42:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d8eb4640 1 --2- 172.21.15.46:0/4014449822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d414a390 0x7f48d40b4bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48d00097c0 tx=0x7f48d001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acc1c845998232fc server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48c7fff640 1 -- 172.21.15.46:0/4014449822 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d00ad640 con 0x7f48d414a390 2023-12-16T21:42:49.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48c7fff640 1 -- 172.21.15.46:0/4014449822 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d00ad7a0 con 0x7f48d414a390 2023-12-16T21:42:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48c7fff640 1 -- 172.21.15.46:0/4014449822 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d00adb30 con 0x7f48d414a390 2023-12-16T21:42:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- 172.21.15.46:0/4014449822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d414a390 msgr2=0x7f48d40b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 --2- 172.21.15.46:0/4014449822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d414a390 0x7f48d40b4bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f48d00097c0 tx=0x7f48d001c480 comp rx=0 tx=0).stop 2023-12-16T21:42:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- 172.21.15.46:0/4014449822 shutdown_connections 2023-12-16T21:42:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 --2- 172.21.15.46:0/4014449822 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d413ff50 0x7f48d4140390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 --2- 172.21.15.46:0/4014449822 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d414a390 0x7f48d40b4bb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- 172.21.15.46:0/4014449822 >> 172.21.15.46:0/4014449822 conn(0x7f48d40722b0 msgr2=0x7f48d40bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- 172.21.15.46:0/4014449822 shutdown_connections 2023-12-16T21:42:49.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- 172.21.15.46:0/4014449822 wait complete. 2023-12-16T21:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 Processor -- start 2023-12-16T21:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- start start 2023-12-16T21:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d40bc800 0x7f48d4150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d413ff50 0x7f48d414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d414a390 0x7f48d414bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d413d040 con 0x7f48d40bc800 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d41dec30 con 0x7f48d413ff50 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48db919640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d414f7d0 con 0x7f48d414a390 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d40bc800 0x7f48d4150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d40bc800 0x7f48d4150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51374/0 (socket says 172.21.15.46:51374) 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 -- 172.21.15.46:0/4077784835 learned_addr learned my addr 172.21.15.46:0/4077784835 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d8eb4640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d413ff50 0x7f48d414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d9eb6640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d414a390 0x7f48d414bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 -- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d414a390 msgr2=0x7f48d414bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d414a390 0x7f48d414bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 -- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d413ff50 msgr2=0x7f48d414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d413ff50 0x7f48d414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 -- 172.21.15.46:0/4077784835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d0009420 con 0x7f48d40bc800 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d9eb6640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d414a390 0x7f48d414bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.475+0000 7f48d96b5640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d40bc800 0x7f48d4150230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48c800cba0 tx=0x7f48c800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48c5ffb640 1 -- 172.21.15.46:0/4077784835 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800ad40 con 0x7f48d40bc800 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48c5ffb640 1 -- 172.21.15.46:0/4077784835 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c8012400 con 0x7f48d40bc800 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48d414fab0 con 0x7f48d40bc800 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48c5ffb640 1 -- 172.21.15.46:0/4077784835 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800aea0 con 0x7f48d40bc800 2023-12-16T21:42:49.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48d41df190 con 0x7f48d40bc800 2023-12-16T21:42:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48d4067de0 con 0x7f48d40bc800 2023-12-16T21:42:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48c5ffb640 1 -- 172.21.15.46:0/4077784835 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f48c8007070 con 0x7f48d40bc800 2023-12-16T21:42:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48c5ffb640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48b8066470 0x7f48b8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48c5ffb640 1 -- 172.21.15.46:0/4077784835 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48c8083ca0 con 0x7f48d40bc800 2023-12-16T21:42:49.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48d8eb4640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48b8066470 0x7f48b8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.479+0000 7f48d8eb4640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48b8066470 0x7f48b8068930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48d00af000 tx=0x7f48d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.483+0000 7f48c5ffb640 1 -- 172.21.15.46:0/4077784835 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f48c8051260 con 0x7f48d40bc800 2023-12-16T21:42:49.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.635+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48d40688f0 con 0x7f48d40bc800 2023-12-16T21:42:49.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48c5ffb640 1 -- 172.21.15.46:0/4077784835 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f48c8054880 con 0x7f48d40bc800 2023-12-16T21:42:49.646 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:49.646 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48b8066470 msgr2=0x7f48b8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48b8066470 0x7f48b8068930 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48d00af000 tx=0x7f48d0008040 comp rx=0 tx=0).stop 2023-12-16T21:42:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d40bc800 msgr2=0x7f48d4150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d40bc800 0x7f48d4150230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f48c800cba0 tx=0x7f48c800a430 comp rx=0 tx=0).stop 2023-12-16T21:42:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 shutdown_connections 2023-12-16T21:42:49.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48b8066470 0x7f48b8068930 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48d414a390 0x7f48d414bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48d413ff50 0x7f48d414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 --2- 172.21.15.46:0/4077784835 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48d40bc800 0x7f48d4150230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 >> 172.21.15.46:0/4077784835 conn(0x7f48d40722b0 msgr2=0x7f48d4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:49.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 shutdown_connections 2023-12-16T21:42:49.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.639+0000 7f48db919640 1 -- 172.21.15.46:0/4077784835 wait complete. 2023-12-16T21:42:49.663 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:49.664 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.775+0000 7f5410d52640 1 Processor -- start 2023-12-16T21:42:49.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.775+0000 7f5410d52640 1 -- start start 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5410d52640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f540c0aded0 0x7f540c143160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5410d52640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c0ace20 con 0x7f540c1436a0 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5410d52640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f540c0aeee0 con 0x7f540c0b3560 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5410d52640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f540c143b50 con 0x7f540c0aded0 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f540c1436a0 0x7f540c0b2e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42032/0 (socket says 172.21.15.46:42032) 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 -- 172.21.15.46:0/2051034402 learned_addr learned my addr 172.21.15.46:0/2051034402 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540a59c640 1 --2- 172.21.15.46:0/2051034402 >> v2:172.21.15.161:3300/0 conn(0x7f540c0aded0 0x7f540c143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942089776 0 0) 0x7f540c0ace20 con 0x7f540c1436a0 2023-12-16T21:42:49.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f8003530 con 0x7f540c1436a0 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034226424 0 0) 0x7f53f8003530 con 0x7f540c1436a0 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 >> v2:172.21.15.161:3300/0 conn(0x7f540c0aded0 msgr2=0x7f540c143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 --2- 172.21.15.46:0/2051034402 >> v2:172.21.15.161:3300/0 conn(0x7f540c0aded0 0x7f540c143160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 >> v1:172.21.15.161:6789/0 conn(0x7f540c0b3560 legacy=0x7f540c14d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c0ad070 con 0x7f540c1436a0 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f53fc001680 con 0x7f540c1436a0 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 >> v1:172.21.15.46:6789/0 conn(0x7f540c1436a0 legacy=0x7f540c0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f8003980 0x7f53f8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53f80047d0 0x7f53f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80052b0 0x7f53f800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c143b50 con 0x7f53f80047d0 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f800c180 con 0x7f53f8003980 2023-12-16T21:42:49.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f800c340 con 0x7f53f80052b0 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53f80047d0 0x7f53f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540a59c640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f8003980 0x7f53f8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540ad9d640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80052b0 0x7f53f800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 -- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80052b0 msgr2=0x7f53f800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80052b0 0x7f53f800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 -- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f8003980 msgr2=0x7f53f8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f53f8003980 0x7f53f8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f5409d9b640 1 -- 172.21.15.46:0/2051034402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f8003530 con 0x7f53f80047d0 2023-12-16T21:42:49.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540ad9d640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80052b0 0x7f53f800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:49.791 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.779+0000 7f540ad9d640 1 -- 172.21.15.46:0/2051034402 reap_dead start 2023-12-16T21:42:49.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5409d9b640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53f80047d0 0x7f53f8004c50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c190 tx=0x7f53fc01cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d48bb54cee45dea server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc005ae0 con 0x7f53f80047d0 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc005c40 con 0x7f53f80047d0 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f540959a640 1 -- 172.21.15.46:0/2051034402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a5a10 con 0x7f53f80047d0 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- 172.21.15.46:0/2051034402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53f80047d0 msgr2=0x7f53f8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53f80047d0 0x7f53f8004c50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53fc00c190 tx=0x7f53fc01cbe0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- 172.21.15.46:0/2051034402 shutdown_connections 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 --2- 172.21.15.46:0/2051034402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53f80047d0 0x7f53f8004c50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- 172.21.15.46:0/2051034402 >> 172.21.15.46:0/2051034402 conn(0x7f540c0b4900 msgr2=0x7f540c0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:49.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- 172.21.15.46:0/2051034402 shutdown_connections 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- 172.21.15.46:0/2051034402 wait complete. 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 Processor -- start 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- start start 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80047d0 0x7f540c070610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f540c1e07b0 0x7f540c1e0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f540c150b10 0x7f540c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c1e1170 con 0x7f540c1e07b0 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540c09a010 con 0x7f540c150b10 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5410d52640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f540c146b50 con 0x7f53f80047d0 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5409d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f540c1e07b0 0x7f540c1e0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5409d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f540c1e07b0 0x7f540c1e0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51382/0 (socket says 172.21.15.46:51382) 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f5409d9b640 1 -- 172.21.15.46:0/1813869411 learned_addr learned my addr 172.21.15.46:0/1813869411 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.783+0000 7f540a59c640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80047d0 0x7f540c070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f540ad9d640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f540c150b10 0x7f540c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5409d9b640 1 -- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80047d0 msgr2=0x7f540c070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5409d9b640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80047d0 0x7f540c070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5409d9b640 1 -- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f540c150b10 msgr2=0x7f540c150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5409d9b640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f540c150b10 0x7f540c150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5409d9b640 1 -- 172.21.15.46:0/1813869411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f540c146e30 con 0x7f540c1e07b0 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f540ad9d640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f540c150b10 0x7f540c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5409d9b640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f540c1e07b0 0x7f540c1e0c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f53fc01ca10 tx=0x7f53f8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5402ffd640 1 -- 172.21.15.46:0/1813869411 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0b4070 con 0x7f540c1e07b0 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5402ffd640 1 -- 172.21.15.46:0/1813869411 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc0a57d0 con 0x7f540c1e07b0 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5402ffd640 1 -- 172.21.15.46:0/1813869411 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc0a5b60 con 0x7f540c1e07b0 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540c099d30 con 0x7f540c1e07b0 2023-12-16T21:42:49.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540c070ce0 con 0x7f540c1e07b0 2023-12-16T21:42:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f540c1e0600 con 0x7f540c1e07b0 2023-12-16T21:42:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5402ffd640 1 -- 172.21.15.46:0/1813869411 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f53fc0a5cc0 con 0x7f540c1e07b0 2023-12-16T21:42:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5402ffd640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f53ec066470 0x7f53ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f5402ffd640 1 -- 172.21.15.46:0/1813869411 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53fc11dc60 con 0x7f540c1e07b0 2023-12-16T21:42:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f540a59c640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f53ec066470 0x7f53ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:49.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.787+0000 7f540a59c640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f53ec066470 0x7f53ec068930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f540c071d50 tx=0x7f53f4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:49.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.791+0000 7f5402ffd640 1 -- 172.21.15.46:0/1813869411 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f53fc0eb170 con 0x7f540c1e07b0 2023-12-16T21:42:49.956 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.947+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f540c068960 con 0x7f540c1e07b0 2023-12-16T21:42:49.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5402ffd640 1 -- 172.21.15.46:0/1813869411 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53fc0ee790 con 0x7f540c1e07b0 2023-12-16T21:42:49.957 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f53ec066470 msgr2=0x7f53ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f53ec066470 0x7f53ec068930 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f540c071d50 tx=0x7f53f4008040 comp rx=0 tx=0).stop 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f540c1e07b0 msgr2=0x7f540c1e0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f540c1e07b0 0x7f540c1e0c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f53fc01ca10 tx=0x7f53f8005800 comp rx=0 tx=0).stop 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 shutdown_connections 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f53ec066470 0x7f53ec068930 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f540c150b10 0x7f540c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f540c1e07b0 0x7f540c1e0c30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 --2- 172.21.15.46:0/1813869411 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f53f80047d0 0x7f540c070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 >> 172.21.15.46:0/1813869411 conn(0x7f540c0b4900 msgr2=0x7f540c0b39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:49.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 shutdown_connections 2023-12-16T21:42:49.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:49.951+0000 7f5410d52640 1 -- 172.21.15.46:0/1813869411 wait complete. 2023-12-16T21:42:49.976 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:42:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 Processor -- start 2023-12-16T21:42:50.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 -- start start 2023-12-16T21:42:50.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f50a0071d70 0x7f50a0147a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a00b3fc0 0x7f50a00b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f50a013e180 0x7f50a013e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a00b4420 con 0x7f50a0071d70 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50a00b3850 con 0x7f50a013e180 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a8155640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a00b3b30 con 0x7f50a00b3fc0 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f50a0071d70 0x7f50a0147a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f50a0071d70 0x7f50a0147a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51390/0 (socket says 172.21.15.46:51390) 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 -- 172.21.15.46:0/3167420189 learned_addr learned my addr 172.21.15.46:0/3167420189 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a66f2640 1 --2- 172.21.15.46:0/3167420189 >> v2:172.21.15.161:3300/0 conn(0x7f50a013e180 0x7f50a013e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a56f0640 1 --2- 172.21.15.46:0/3167420189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a00b3fc0 0x7f50a00b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 -- 172.21.15.46:0/3167420189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a00b3fc0 msgr2=0x7f50a00b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:50.097 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 --2- 172.21.15.46:0/3167420189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a00b3fc0 0x7f50a00b4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 -- 172.21.15.46:0/3167420189 >> v2:172.21.15.161:3300/0 conn(0x7f50a013e180 msgr2=0x7f50a013e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 --2- 172.21.15.46:0/3167420189 >> v2:172.21.15.161:3300/0 conn(0x7f50a013e180 0x7f50a013e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a5ef1640 1 -- 172.21.15.46:0/3167420189 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a0072150 con 0x7f50a0071d70 2023-12-16T21:42:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a66f2640 1 --2- 172.21.15.46:0/3167420189 >> v2:172.21.15.161:3300/0 conn(0x7f50a013e180 0x7f50a013e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:50.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.087+0000 7f50a56f0640 1 --2- 172.21.15.46:0/3167420189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a00b3fc0 0x7f50a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a5ef1640 1 --2- 172.21.15.46:0/3167420189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a0071d70 0x7f50a0147a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50a00b3aa0 tx=0x7f509001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b56f7e507fe86f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a4eef640 1 -- 172.21.15.46:0/3167420189 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900ab600 con 0x7f50a0071d70 2023-12-16T21:42:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a4eef640 1 -- 172.21.15.46:0/3167420189 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50900a7050 con 0x7f50a0071d70 2023-12-16T21:42:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a4eef640 1 -- 172.21.15.46:0/3167420189 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50900ab980 con 0x7f50a0071d70 2023-12-16T21:42:50.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 -- 172.21.15.46:0/3167420189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a0071d70 msgr2=0x7f50a0147a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 --2- 172.21.15.46:0/3167420189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a0071d70 0x7f50a0147a20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50a00b3aa0 tx=0x7f509001c470 comp rx=0 tx=0).stop 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 -- 172.21.15.46:0/3167420189 shutdown_connections 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 --2- 172.21.15.46:0/3167420189 >> v2:172.21.15.161:3300/0 conn(0x7f50a013e180 0x7f50a013e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 --2- 172.21.15.46:0/3167420189 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a00b3fc0 0x7f50a00b4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 --2- 172.21.15.46:0/3167420189 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a0071d70 0x7f50a0147a20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 -- 172.21.15.46:0/3167420189 >> 172.21.15.46:0/3167420189 conn(0x7f50a00b9250 msgr2=0x7f50a00b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 -- 172.21.15.46:0/3167420189 shutdown_connections 2023-12-16T21:42:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 -- 172.21.15.46:0/3167420189 wait complete. 2023-12-16T21:42:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 Processor -- start 2023-12-16T21:42:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.091+0000 7f50a8155640 1 -- start start 2023-12-16T21:42:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50a0071d70 0x7f50a013c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00b3fc0 0x7f50a01354a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a013e180 0x7f50a0131b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a013f960 con 0x7f50a00b3fc0 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a0134d10 con 0x7f50a0071d70 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a013feb0 con 0x7f50a013e180 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00b3fc0 0x7f50a01354a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00b3fc0 0x7f50a01354a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51400/0 (socket says 172.21.15.46:51400) 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 -- 172.21.15.46:0/3620640162 learned_addr learned my addr 172.21.15.46:0/3620640162 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a5ef1640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50a0071d70 0x7f50a013c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a66f2640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a013e180 0x7f50a0131b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 -- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a013e180 msgr2=0x7f50a0131b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a013e180 0x7f50a0131b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 -- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50a0071d70 msgr2=0x7f50a013c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50a0071d70 0x7f50a013c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 -- 172.21.15.46:0/3620640162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50900094d0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.103 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a66f2640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a013e180 0x7f50a0131b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a56f0640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00b3fc0 0x7f50a01354a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c7f0 tx=0x7f509c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50967fc640 1 -- 172.21.15.46:0/3620640162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c00cea0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50967fc640 1 -- 172.21.15.46:0/3620640162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c007a30 con 0x7f50a00b3fc0 2023-12-16T21:42:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50967fc640 1 -- 172.21.15.46:0/3620640162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c007de0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a0134f80 con 0x7f50a00b3fc0 2023-12-16T21:42:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a013d040 con 0x7f50a00b3fc0 2023-12-16T21:42:50.104 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a0068960 con 0x7f50a00b3fc0 2023-12-16T21:42:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50967fc640 1 -- 172.21.15.46:0/3620640162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f509c0113f0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50967fc640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5084066470 0x7f5084068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50967fc640 1 -- 172.21.15.46:0/3620640162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f509c083ec0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.095+0000 7f50a5ef1640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5084066470 0x7f5084068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.099+0000 7f50a5ef1640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5084066470 0x7f5084068930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50900ad000 tx=0x7f5090008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.099+0000 7f50967fc640 1 -- 172.21.15.46:0/3620640162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f509c0513d0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.255+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f50a00b47c0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.263 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.255+0000 7f50967fc640 1 -- 172.21.15.46:0/3620640162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f509c0549f0 con 0x7f50a00b3fc0 2023-12-16T21:42:50.263 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5084066470 msgr2=0x7f5084068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5084066470 0x7f5084068930 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f50900ad000 tx=0x7f5090008040 comp rx=0 tx=0).stop 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00b3fc0 msgr2=0x7f50a01354a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00b3fc0 0x7f50a01354a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f509c00c7f0 tx=0x7f509c00ccb0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 shutdown_connections 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5084066470 0x7f5084068930 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f50a013e180 0x7f50a0131b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f50a00b3fc0 0x7f50a01354a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 --2- 172.21.15.46:0/3620640162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f50a0071d70 0x7f50a013c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 >> 172.21.15.46:0/3620640162 conn(0x7f50a00b9250 msgr2=0x7f50a00b2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 shutdown_connections 2023-12-16T21:42:50.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.259+0000 7f50a8155640 1 -- 172.21.15.46:0/3620640162 wait complete. 2023-12-16T21:42:50.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:42:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.391+0000 7f42af639640 1 Processor -- start 2023-12-16T21:42:50.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.391+0000 7f42af639640 1 -- start start 2023-12-16T21:42:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42af639640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f42a80b1010 0x7f42a813c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42af639640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a80b4820 0x7f42a813cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42af639640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a80b3ae0 con 0x7f42a80b3700 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42af639640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a813d240 con 0x7f42a80b1010 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42af639640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b1420 con 0x7f42a80b4820 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42a80b3700 0x7f42a8142a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42040/0 (socket says 172.21.15.46:42040) 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 -- 172.21.15.46:0/942085708 learned_addr learned my addr 172.21.15.46:0/942085708 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42acbd4640 1 --2- 172.21.15.46:0/942085708 >> v2:172.21.15.161:3300/0 conn(0x7f42a80b1010 0x7f42a813c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42adbd6640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a80b4820 0x7f42a813cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183818914 0 0) 0x7f42a80b3ae0 con 0x7f42a80b3700 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4294004640 con 0x7f42a80b3700 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3195623793 0 0) 0x7f4294004640 con 0x7f42a80b3700 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a80b4820 msgr2=0x7f42a813cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a80b4820 0x7f42a813cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 >> v2:172.21.15.161:3300/0 conn(0x7f42a80b1010 msgr2=0x7f42a813c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 --2- 172.21.15.46:0/942085708 >> v2:172.21.15.161:3300/0 conn(0x7f42a80b1010 0x7f42a813c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80b1630 con 0x7f42a80b3700 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f42a4001680 con 0x7f42a80b3700 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 >> v1:172.21.15.46:6789/0 conn(0x7f42a80b3700 legacy=0x7f42a8142a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4294004a90 0x7f4294004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a80b4820 0x7f42940058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294005e20 0x7f42940062a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b1420 con 0x7f4294004a90 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429400caa0 con 0x7f4294005e20 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429400cc10 con 0x7f42a80b4820 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4294004a90 0x7f4294004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42adbd6640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294005e20 0x7f42940062a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 -- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a80b4820 msgr2=0x7f42940058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a80b4820 0x7f42940058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 -- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294005e20 msgr2=0x7f42940062a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294005e20 0x7f42940062a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 -- 172.21.15.46:0/942085708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294004640 con 0x7f4294004a90 2023-12-16T21:42:50.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42adbd6640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294005e20 0x7f42940062a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f42ad3d5640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4294004a90 0x7f4294004eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c110 tx=0x7f42a401cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e24892a38c07c802 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a40058e0 con 0x7f4294004a90 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a4005a60 con 0x7f4294004a90 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.395+0000 7f429ffff640 1 -- 172.21.15.46:0/942085708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a4005e10 con 0x7f4294004a90 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/942085708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4294004a90 msgr2=0x7f4294004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 --2- 172.21.15.46:0/942085708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4294004a90 0x7f4294004eb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f42a400c110 tx=0x7f42a401cb70 comp rx=0 tx=0).stop 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 -- 172.21.15.46:0/942085708 reap_dead start 2023-12-16T21:42:50.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/942085708 shutdown_connections 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/942085708 >> 172.21.15.46:0/942085708 conn(0x7f42a80b8210 msgr2=0x7f42a80b8620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/942085708 shutdown_connections 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/942085708 wait complete. 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 Processor -- start 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- start start 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294004a90 0x7f42a80aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a8070b50 0x7f42a806fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a81e1cf0 0x7f42a81e2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a806f680 con 0x7f42a81e1cf0 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8071ad0 con 0x7f4294004a90 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a806f430 con 0x7f42a8070b50 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a81e1cf0 0x7f42a81e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a81e1cf0 0x7f42a81e2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51432/0 (socket says 172.21.15.46:51432) 2023-12-16T21:42:50.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 -- 172.21.15.46:0/1325034885 learned_addr learned my addr 172.21.15.46:0/1325034885 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42ad3d5640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294004a90 0x7f42a80aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42acbd4640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a8070b50 0x7f42a806fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 -- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a8070b50 msgr2=0x7f42a806fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a8070b50 0x7f42a806fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 -- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294004a90 msgr2=0x7f42a80aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294004a90 0x7f42a80aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 -- 172.21.15.46:0/1325034885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8072030 con 0x7f42a81e1cf0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42adbd6640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a81e1cf0 0x7f42a81e2170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f42a80741c0 tx=0x7f42a000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f429dffb640 1 -- 172.21.15.46:0/1325034885 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a000fec0 con 0x7f42a81e1cf0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f429dffb640 1 -- 172.21.15.46:0/1325034885 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a0011040 con 0x7f42a81e1cf0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f429dffb640 1 -- 172.21.15.46:0/1325034885 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0017600 con 0x7f42a81e1cf0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a8071d00 con 0x7f42a81e1cf0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a80af650 con 0x7f42a81e1cf0 2023-12-16T21:42:50.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.399+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a80721c0 con 0x7f42a81e1cf0 2023-12-16T21:42:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.403+0000 7f429dffb640 1 -- 172.21.15.46:0/1325034885 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f42a00203b0 con 0x7f42a81e1cf0 2023-12-16T21:42:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.403+0000 7f429dffb640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f428c066470 0x7f428c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.403+0000 7f429dffb640 1 -- 172.21.15.46:0/1325034885 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42a0083fc0 con 0x7f42a81e1cf0 2023-12-16T21:42:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.403+0000 7f42ad3d5640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f428c066470 0x7f428c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.403+0000 7f42ad3d5640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f428c066470 0x7f428c068930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f42a40aa000 tx=0x7f42a40019b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.407+0000 7f429dffb640 1 -- 172.21.15.46:0/1325034885 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f42a0054ec0 con 0x7f42a81e1cf0 2023-12-16T21:42:50.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.555+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f42a8068960 con 0x7f42a81e1cf0 2023-12-16T21:42:50.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.555+0000 7f429dffb640 1 -- 172.21.15.46:0/1325034885 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f42a0054a50 con 0x7f42a81e1cf0 2023-12-16T21:42:50.564 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:42:50.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f428c066470 msgr2=0x7f428c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f428c066470 0x7f428c068930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f42a40aa000 tx=0x7f42a40019b0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a81e1cf0 msgr2=0x7f42a81e2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a81e1cf0 0x7f42a81e2170 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f42a80741c0 tx=0x7f42a000f4a0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 shutdown_connections 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f428c066470 0x7f428c068930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42a81e1cf0 0x7f42a81e2170 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42a8070b50 0x7f42a806fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 --2- 172.21.15.46:0/1325034885 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4294004a90 0x7f42a80aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 >> 172.21.15.46:0/1325034885 conn(0x7f42a80b8210 msgr2=0x7f42a80b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 shutdown_connections 2023-12-16T21:42:50.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.559+0000 7f42af639640 1 -- 172.21.15.46:0/1325034885 wait complete. 2023-12-16T21:42:50.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:42:50.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.691+0000 7f8dda142640 1 Processor -- start 2023-12-16T21:42:50.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.691+0000 7f8dda142640 1 -- start start 2023-12-16T21:42:50.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dda142640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8dd4146560 0x7f8dd4146d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dda142640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8dd40b17a0 0x7f8dd40b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dda142640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40ae7e0 con 0x7f8dd40b27e0 2023-12-16T21:42:50.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dda142640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd40afb60 con 0x7f8dd4146560 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dda142640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd40aeda0 con 0x7f8dd40b17a0 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8dd4146560 0x7f8dd4146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8dd4146560 0x7f8dd4146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51446/0 (socket says 172.21.15.46:51446) 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd37fe640 1 -- 172.21.15.46:0/3988225251 learned_addr learned my addr 172.21.15.46:0/3988225251 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd2ffd640 1 --2- 172.21.15.46:0/3988225251 >> v2:172.21.15.161:3300/0 conn(0x7f8dd40b17a0 0x7f8dd40b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd37fe640 1 -- 172.21.15.46:0/3988225251 >> v2:172.21.15.161:3300/0 conn(0x7f8dd40b17a0 msgr2=0x7f8dd40b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd37fe640 1 --2- 172.21.15.46:0/3988225251 >> v2:172.21.15.161:3300/0 conn(0x7f8dd40b17a0 0x7f8dd40b1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd37fe640 1 -- 172.21.15.46:0/3988225251 >> v1:172.21.15.161:6789/0 conn(0x7f8dd40b27e0 legacy=0x7f8dd40b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd37fe640 1 -- 172.21.15.46:0/3988225251 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40afd60 con 0x7f8dd4146560 2023-12-16T21:42:50.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.695+0000 7f8dd2ffd640 1 --2- 172.21.15.46:0/3988225251 >> v2:172.21.15.161:3300/0 conn(0x7f8dd40b17a0 0x7f8dd40b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd37fe640 1 --2- 172.21.15.46:0/3988225251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd4146560 0x7f8dd4146d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800bb10 tx=0x7f8dc801eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75dc64b1d93fbe3 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd27fc640 1 -- 172.21.15.46:0/3988225251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80a7b00 con 0x7f8dd4146560 2023-12-16T21:42:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd27fc640 1 -- 172.21.15.46:0/3988225251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc80a7c60 con 0x7f8dd4146560 2023-12-16T21:42:50.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd27fc640 1 -- 172.21.15.46:0/3988225251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80af440 con 0x7f8dd4146560 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- 172.21.15.46:0/3988225251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd4146560 msgr2=0x7f8dd4146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 --2- 172.21.15.46:0/3988225251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd4146560 0x7f8dd4146d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800bb10 tx=0x7f8dc801eab0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- 172.21.15.46:0/3988225251 shutdown_connections 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 --2- 172.21.15.46:0/3988225251 >> v2:172.21.15.161:3300/0 conn(0x7f8dd40b17a0 0x7f8dd40b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 --2- 172.21.15.46:0/3988225251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd4146560 0x7f8dd4146d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- 172.21.15.46:0/3988225251 >> 172.21.15.46:0/3988225251 conn(0x7f8dd40b48e0 msgr2=0x7f8dd40b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- 172.21.15.46:0/3988225251 shutdown_connections 2023-12-16T21:42:50.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- 172.21.15.46:0/3988225251 wait complete. 2023-12-16T21:42:50.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 Processor -- start 2023-12-16T21:42:50.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- start start 2023-12-16T21:42:50.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd40b17a0 0x7f8dd413b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd40b27e0 0x7f8dd41418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd4146560 0x7f8dd4141e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41daa50 con 0x7f8dd40b27e0 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd414e320 con 0x7f8dd40b17a0 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41dac40 con 0x7f8dd4146560 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd40b27e0 0x7f8dd41418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd40b27e0 0x7f8dd41418d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51462/0 (socket says 172.21.15.46:51462) 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 -- 172.21.15.46:0/1076761826 learned_addr learned my addr 172.21.15.46:0/1076761826 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd37fe640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd40b17a0 0x7f8dd413b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd3fff640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd4146560 0x7f8dd4141e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 -- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd4146560 msgr2=0x7f8dd4141e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd4146560 0x7f8dd4141e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 -- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd40b17a0 msgr2=0x7f8dd413b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd40b17a0 0x7f8dd413b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 -- 172.21.15.46:0/1076761826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc800b770 con 0x7f8dd40b27e0 2023-12-16T21:42:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dd2ffd640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd40b27e0 0x7f8dd41418d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c7f0 tx=0x7f8dc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dbbfff640 1 -- 172.21.15.46:0/1076761826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc40077c0 con 0x7f8dd40b27e0 2023-12-16T21:42:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dbbfff640 1 -- 172.21.15.46:0/1076761826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4007920 con 0x7f8dd40b27e0 2023-12-16T21:42:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dbbfff640 1 -- 172.21.15.46:0/1076761826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4007cd0 con 0x7f8dd40b27e0 2023-12-16T21:42:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd41daf20 con 0x7f8dd40b27e0 2023-12-16T21:42:50.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.699+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd4143d90 con 0x7f8dd40b27e0 2023-12-16T21:42:50.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.703+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd40679e0 con 0x7f8dd40b27e0 2023-12-16T21:42:50.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.703+0000 7f8dbbfff640 1 -- 172.21.15.46:0/1076761826 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8dc40113f0 con 0x7f8dd40b27e0 2023-12-16T21:42:50.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.703+0000 7f8dbbfff640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dc00664a0 0x7f8dc0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:50.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.703+0000 7f8dbbfff640 1 -- 172.21.15.46:0/1076761826 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dc4084090 con 0x7f8dd40b27e0 2023-12-16T21:42:50.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.703+0000 7f8dd37fe640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dc00664a0 0x7f8dc0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:50.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.703+0000 7f8dd37fe640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dc00664a0 0x7f8dc0068960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80b1000 tx=0x7f8dc80a9070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:50.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.707+0000 7f8dbbfff640 1 -- 172.21.15.46:0/1076761826 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8dc40515a0 con 0x7f8dd40b27e0 2023-12-16T21:42:50.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.887+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dd40688f0 con 0x7f8dd40b27e0 2023-12-16T21:42:50.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.887+0000 7f8dbbfff640 1 -- 172.21.15.46:0/1076761826 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7f8dc401fd30 con 0x7f8dd40b27e0 2023-12-16T21:42:50.895 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:50.895 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:42:50.895 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dc00664a0 msgr2=0x7f8dc0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dc00664a0 0x7f8dc0068960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80b1000 tx=0x7f8dc80a9070 comp rx=0 tx=0).stop 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd40b27e0 msgr2=0x7f8dd41418d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd40b27e0 0x7f8dd41418d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c7f0 tx=0x7f8dc400ccb0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 shutdown_connections 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8dc00664a0 0x7f8dc0068960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8dd4146560 0x7f8dd4141e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8dd40b27e0 0x7f8dd41418d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 --2- 172.21.15.46:0/1076761826 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8dd40b17a0 0x7f8dd413b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 >> 172.21.15.46:0/1076761826 conn(0x7f8dd40b48e0 msgr2=0x7f8dd40b74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 shutdown_connections 2023-12-16T21:42:50.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:50.891+0000 7f8dda142640 1 -- 172.21.15.46:0/1076761826 wait complete. 2023-12-16T21:42:50.915 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:51.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccf9de640 1 Processor -- start 2023-12-16T21:42:51.029 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccf9de640 1 -- start start 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccf9de640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2cc80b3fd0 0x7f2cc813e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccf9de640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc80b0340 0x7f2cc813e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccf9de640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc80af2e0 con 0x7f2cc806c920 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccf9de640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc80b1f50 con 0x7f2cc80b3fd0 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccf9de640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc813eda0 con 0x7f2cc80b0340 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cccf79640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2cc80b3fd0 0x7f2cc813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccd77a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2cc806c920 0x7f2cc80afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42054/0 (socket says 172.21.15.46:42054) 2023-12-16T21:42:51.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccd77a640 1 -- 172.21.15.46:0/1604195277 learned_addr learned my addr 172.21.15.46:0/1604195277 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2ccdf7b640 1 --2- 172.21.15.46:0/1604195277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc80b0340 0x7f2cc813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cccf79640 1 -- 172.21.15.46:0/1604195277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc80b0340 msgr2=0x7f2cc813e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cccf79640 1 --2- 172.21.15.46:0/1604195277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc80b0340 0x7f2cc813e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cccf79640 1 -- 172.21.15.46:0/1604195277 >> v1:172.21.15.46:6789/0 conn(0x7f2cc806c920 legacy=0x7f2cc80afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cccf79640 1 -- 172.21.15.46:0/1604195277 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc806cd00 con 0x7f2cc80b3fd0 2023-12-16T21:42:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cccf79640 1 --2- 172.21.15.46:0/1604195277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc80b3fd0 0x7f2cc813e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40097c0 tx=0x7f2cc401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e198e33a729d2b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cbffff640 1 -- 172.21.15.46:0/1604195277 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40ab600 con 0x7f2cc80b3fd0 2023-12-16T21:42:51.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cbffff640 1 -- 172.21.15.46:0/1604195277 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc40a7050 con 0x7f2cc80b3fd0 2023-12-16T21:42:51.033 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.023+0000 7f2cbffff640 1 -- 172.21.15.46:0/1604195277 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40ab980 con 0x7f2cc80b3fd0 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- 172.21.15.46:0/1604195277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc80b3fd0 msgr2=0x7f2cc813e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/1604195277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc80b3fd0 0x7f2cc813e320 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40097c0 tx=0x7f2cc401c3f0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- 172.21.15.46:0/1604195277 shutdown_connections 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/1604195277 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc80b0340 0x7f2cc813e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/1604195277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc80b3fd0 0x7f2cc813e320 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- 172.21.15.46:0/1604195277 >> 172.21.15.46:0/1604195277 conn(0x7f2cc80b48e0 msgr2=0x7f2cc80b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- 172.21.15.46:0/1604195277 shutdown_connections 2023-12-16T21:42:51.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- 172.21.15.46:0/1604195277 wait complete. 2023-12-16T21:42:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 Processor -- start 2023-12-16T21:42:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- start start 2023-12-16T21:42:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc806c920 0x7f2cc8134210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc8134750 0x7f2cc81417f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc8141d30 0x7f2cc813f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8135700 con 0x7f2cc8134750 2023-12-16T21:42:51.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8140570 con 0x7f2cc806c920 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8141060 con 0x7f2cc8141d30 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc8134750 0x7f2cc81417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc8134750 0x7f2cc81417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51490/0 (socket says 172.21.15.46:51490) 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 -- 172.21.15.46:0/4025500222 learned_addr learned my addr 172.21.15.46:0/4025500222 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccd77a640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc806c920 0x7f2cc8134210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccdf7b640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc8141d30 0x7f2cc813f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 -- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc8141d30 msgr2=0x7f2cc813f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc8141d30 0x7f2cc813f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 -- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc806c920 msgr2=0x7f2cc8134210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc806c920 0x7f2cc8134210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 -- 172.21.15.46:0/4025500222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4009420 con 0x7f2cc8134750 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccd77a640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc806c920 0x7f2cc8134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cccf79640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc8134750 0x7f2cc81417f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401c8d0 tx=0x7f2cc4019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cbdffb640 1 -- 172.21.15.46:0/4025500222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40b4810 con 0x7f2cc8134750 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cbdffb640 1 -- 172.21.15.46:0/4025500222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc40b4970 con 0x7f2cc8134750 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc81412d0 con 0x7f2cc8134750 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2cbdffb640 1 -- 172.21.15.46:0/4025500222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40b4d20 con 0x7f2cc8134750 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.027+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc813d0c0 con 0x7f2cc8134750 2023-12-16T21:42:51.039 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.031+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc80679e0 con 0x7f2cc8134750 2023-12-16T21:42:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.031+0000 7f2cbdffb640 1 -- 172.21.15.46:0/4025500222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2cc40b2040 con 0x7f2cc8134750 2023-12-16T21:42:51.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.031+0000 7f2cbdffb640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cac0664a0 0x7f2cac068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.031+0000 7f2cbdffb640 1 -- 172.21.15.46:0/4025500222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cc411e220 con 0x7f2cc8134750 2023-12-16T21:42:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.031+0000 7f2ccd77a640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cac0664a0 0x7f2cac068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.031+0000 7f2ccd77a640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cac0664a0 0x7f2cac068960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8007fd0 tx=0x7f2cb8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.035+0000 7f2cbdffb640 1 -- 172.21.15.46:0/4025500222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2cc40eb690 con 0x7f2cc8134750 2023-12-16T21:42:51.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.187+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cc813cf20 con 0x7f2cc8134750 2023-12-16T21:42:51.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2cbdffb640 1 -- 172.21.15.46:0/4025500222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f2cc813cf20 con 0x7f2cc8134750 2023-12-16T21:42:51.197 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:51.197 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cac0664a0 msgr2=0x7f2cac068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cac0664a0 0x7f2cac068960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8007fd0 tx=0x7f2cb8009040 comp rx=0 tx=0).stop 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc8134750 msgr2=0x7f2cc81417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc8134750 0x7f2cc81417f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2cc401c8d0 tx=0x7f2cc4019190 comp rx=0 tx=0).stop 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 shutdown_connections 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2cac0664a0 0x7f2cac068960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2cc8141d30 0x7f2cc813f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2cc8134750 0x7f2cc81417f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 --2- 172.21.15.46:0/4025500222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2cc806c920 0x7f2cc8134210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 >> 172.21.15.46:0/4025500222 conn(0x7f2cc80b48e0 msgr2=0x7f2cc80b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 shutdown_connections 2023-12-16T21:42:51.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.191+0000 7f2ccf9de640 1 -- 172.21.15.46:0/4025500222 wait complete. 2023-12-16T21:42:51.215 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:42:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3081372640 1 Processor -- start 2023-12-16T21:42:51.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3081372640 1 -- start start 2023-12-16T21:42:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3081372640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f307c0ba740 0x7f307c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3081372640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0b20f0 con 0x7f307c0b63c0 2023-12-16T21:42:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3081372640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f307c0bab20 con 0x7f307c0b5f30 2023-12-16T21:42:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3081372640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f307c0b09a0 con 0x7f307c0ba740 2023-12-16T21:42:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307b7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f307c0b63c0 0x7f307c14ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42064/0 (socket says 172.21.15.46:42064) 2023-12-16T21:42:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307b7fe640 1 -- 172.21.15.46:0/385452735 learned_addr learned my addr 172.21.15.46:0/385452735 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:51.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307affd640 1 --2- 172.21.15.46:0/385452735 >> v2:172.21.15.161:3300/0 conn(0x7f307c0ba740 0x7f307c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402808877 0 0) 0x7f307c0b20f0 con 0x7f307c0b63c0 2023-12-16T21:42:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3064003530 con 0x7f307c0b63c0 2023-12-16T21:42:51.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108760548 0 0) 0x7f3064003530 con 0x7f307c0b63c0 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 >> v2:172.21.15.161:3300/0 conn(0x7f307c0ba740 msgr2=0x7f307c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 --2- 172.21.15.46:0/385452735 >> v2:172.21.15.161:3300/0 conn(0x7f307c0ba740 0x7f307c0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 >> v1:172.21.15.161:6789/0 conn(0x7f307c0b5f30 legacy=0x7f307c0b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0b0c80 con 0x7f307c0b63c0 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f3074001a00 con 0x7f307c0b63c0 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 >> v1:172.21.15.46:6789/0 conn(0x7f307c0b63c0 legacy=0x7f307c14ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3064003980 0x7f3064003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3064004380 0x7f3064004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064005050 0x7f306400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c0b09a0 con 0x7f3064004380 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306400bfb0 con 0x7f3064003980 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306400c170 con 0x7f3064005050 2023-12-16T21:42:51.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307a7fc640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3064004380 0x7f3064004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307affd640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3064003980 0x7f3064003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307b7fe640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064005050 0x7f306400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307a7fc640 1 -- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064005050 msgr2=0x7f306400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307a7fc640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064005050 0x7f306400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307a7fc640 1 -- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3064003980 msgr2=0x7f3064003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307a7fc640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3064003980 0x7f3064003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307a7fc640 1 -- 172.21.15.46:0/385452735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3064003530 con 0x7f3064004380 2023-12-16T21:42:51.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.331+0000 7f307b7fe640 1 -- 172.21.15.46:0/385452735 reap_dead start 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3064004380 0x7f3064004850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3070001860 tx=0x7f307001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3d02a6ec0001fb server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700ab600 con 0x7f3064004380 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700a7050 con 0x7f3064004380 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3079ffb640 1 -- 172.21.15.46:0/385452735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700ab9e0 con 0x7f3064004380 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- 172.21.15.46:0/385452735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3064004380 msgr2=0x7f3064004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3064004380 0x7f3064004850 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3070001860 tx=0x7f307001c4c0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- 172.21.15.46:0/385452735 shutdown_connections 2023-12-16T21:42:51.345 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 --2- 172.21.15.46:0/385452735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3064004380 0x7f3064004850 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- 172.21.15.46:0/385452735 >> 172.21.15.46:0/385452735 conn(0x7f307c06ab80 msgr2=0x7f307c0b9dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- 172.21.15.46:0/385452735 shutdown_connections 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- 172.21.15.46:0/385452735 wait complete. 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 Processor -- start 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- start start 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064004380 0x7f307c13f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307c13b770 0x7f307c13c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307c140f60 0x7f307c13ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1dd4b0 con 0x7f307c13b770 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1dcb30 con 0x7f307c140f60 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1423f0 con 0x7f3064004380 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307c13b770 0x7f307c13c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307c13b770 0x7f307c13c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51510/0 (socket says 172.21.15.46:51510) 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 -- 172.21.15.46:0/2716623592 learned_addr learned my addr 172.21.15.46:0/2716623592 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307b7fe640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307c140f60 0x7f307c13ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307affd640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064004380 0x7f307c13f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 -- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064004380 msgr2=0x7f307c13f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064004380 0x7f307c13f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 -- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307c140f60 msgr2=0x7f307c13ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307c140f60 0x7f307c13ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 -- 172.21.15.46:0/2716623592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c1dcdf0 con 0x7f307c13b770 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f307a7fc640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307c13b770 0x7f307c13c070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30700ae040 tx=0x7f3064005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f30637fe640 1 -- 172.21.15.46:0/2716623592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070007240 con 0x7f307c13b770 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f30637fe640 1 -- 172.21.15.46:0/2716623592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700073c0 con 0x7f307c13b770 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c1dd0d0 con 0x7f307c13b770 2023-12-16T21:42:51.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f30637fe640 1 -- 172.21.15.46:0/2716623592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700b4400 con 0x7f307c13b770 2023-12-16T21:42:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.335+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c1413e0 con 0x7f307c13b770 2023-12-16T21:42:51.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.339+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c067ee0 con 0x7f307c13b770 2023-12-16T21:42:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.339+0000 7f30637fe640 1 -- 172.21.15.46:0/2716623592 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f30700b2040 con 0x7f307c13b770 2023-12-16T21:42:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.339+0000 7f30637fe640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f30680664a0 0x7f3068068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.339+0000 7f30637fe640 1 -- 172.21.15.46:0/2716623592 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f307011d880 con 0x7f307c13b770 2023-12-16T21:42:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.339+0000 7f307affd640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f30680664a0 0x7f3068068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.339+0000 7f307affd640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f30680664a0 0x7f3068068960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f306c009ab0 tx=0x7f306c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.343+0000 7f30637fe640 1 -- 172.21.15.46:0/2716623592 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f30700eacf0 con 0x7f307c13b770 2023-12-16T21:42:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f307c1dd5f0 con 0x7f307c13b770 2023-12-16T21:42:51.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f30637fe640 1 -- 172.21.15.46:0/2716623592 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7f30700b0d30 con 0x7f307c13b770 2023-12-16T21:42:51.538 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:51.538 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:42:51.539 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f30680664a0 msgr2=0x7f3068068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f30680664a0 0x7f3068068960 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f306c009ab0 tx=0x7f306c008040 comp rx=0 tx=0).stop 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307c13b770 msgr2=0x7f307c13c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307c13b770 0x7f307c13c070 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30700ae040 tx=0x7f3064005800 comp rx=0 tx=0).stop 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 shutdown_connections 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f30680664a0 0x7f3068068960 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f307c140f60 0x7f307c13ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f307c13b770 0x7f307c13c070 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 --2- 172.21.15.46:0/2716623592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3064004380 0x7f307c13f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 >> 172.21.15.46:0/2716623592 conn(0x7f307c06ab80 msgr2=0x7f307c13e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 shutdown_connections 2023-12-16T21:42:51.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.531+0000 7f3081372640 1 -- 172.21.15.46:0/2716623592 wait complete. 2023-12-16T21:42:51.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-16T21:42:40.776790+0000', 'modified': '2023-12-16T21:42:42.763769+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.161:6835/926962350', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.161:6834', 'nonce': 926962350}, {'type': 'v1', 'addr': '172.21.15.161:6835', 'nonce': 926962350}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-16T21:42:51.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-16T21:42:51.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-16T21:42:51.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1307f640 1 Processor -- start 2023-12-16T21:42:51.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1307f640 1 -- start start 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1307f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e0c06c1a0 0x7f2e0c0b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1307f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2e0c0b1f80 0x7f2e0c14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1307f640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c0af330 con 0x7f2e0c0adf50 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1307f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0c0aed20 con 0x7f2e0c06c1a0 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1307f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e0c06c580 con 0x7f2e0c0b1f80 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e10e1b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e0c06c1a0 0x7f2e0c0b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e10e1b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2e0c06c1a0 0x7f2e0c0b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51516/0 (socket says 172.21.15.46:51516) 2023-12-16T21:42:51.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e10e1b640 1 -- 172.21.15.46:0/1214936971 learned_addr learned my addr 172.21.15.46:0/1214936971 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e1161c640 1 --2- 172.21.15.46:0/1214936971 >> v2:172.21.15.161:3300/0 conn(0x7f2e0c0b1f80 0x7f2e0c14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e10e1b640 1 -- 172.21.15.46:0/1214936971 >> v2:172.21.15.161:3300/0 conn(0x7f2e0c0b1f80 msgr2=0x7f2e0c14d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/1214936971 >> v2:172.21.15.161:3300/0 conn(0x7f2e0c0b1f80 0x7f2e0c14d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e10e1b640 1 -- 172.21.15.46:0/1214936971 >> v1:172.21.15.161:6789/0 conn(0x7f2e0c0adf50 legacy=0x7f2e0c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.879+0000 7f2e10e1b640 1 -- 172.21.15.46:0/1214936971 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c0ad600 con 0x7f2e0c06c1a0 2023-12-16T21:42:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/1214936971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c06c1a0 0x7f2e0c0b2790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c072680 tx=0x7f2dfc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b54d1518dc38f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e0b7fe640 1 -- 172.21.15.46:0/1214936971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0ab600 con 0x7f2e0c06c1a0 2023-12-16T21:42:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e0b7fe640 1 -- 172.21.15.46:0/1214936971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc0ab760 con 0x7f2e0c06c1a0 2023-12-16T21:42:51.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e0b7fe640 1 -- 172.21.15.46:0/1214936971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0abaf0 con 0x7f2e0c06c1a0 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 -- 172.21.15.46:0/1214936971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c06c1a0 msgr2=0x7f2e0c0b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 --2- 172.21.15.46:0/1214936971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c06c1a0 0x7f2e0c0b2790 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c072680 tx=0x7f2dfc01c400 comp rx=0 tx=0).stop 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 -- 172.21.15.46:0/1214936971 shutdown_connections 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 --2- 172.21.15.46:0/1214936971 >> v2:172.21.15.161:3300/0 conn(0x7f2e0c0b1f80 0x7f2e0c14d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 --2- 172.21.15.46:0/1214936971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c06c1a0 0x7f2e0c0b2790 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 -- 172.21.15.46:0/1214936971 >> 172.21.15.46:0/1214936971 conn(0x7f2e0c06a580 msgr2=0x7f2e0c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 -- 172.21.15.46:0/1214936971 shutdown_connections 2023-12-16T21:42:51.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 -- 172.21.15.46:0/1214936971 wait complete. 2023-12-16T21:42:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 Processor -- start 2023-12-16T21:42:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.883+0000 7f2e1307f640 1 -- start start 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e0c06c1a0 0x7f2e0c070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0adf50 0x7f2e0c149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0c0b1f80 0x7f2e0c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1482b0 con 0x7f2e0c0adf50 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c148700 con 0x7f2e0c06c1a0 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c1484c0 con 0x7f2e0c0b1f80 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0adf50 0x7f2e0c149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0adf50 0x7f2e0c149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51524/0 (socket says 172.21.15.46:51524) 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 -- 172.21.15.46:0/2474538481 learned_addr learned my addr 172.21.15.46:0/2474538481 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1161c640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0c0b1f80 0x7f2e0c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e0c06c1a0 0x7f2e0c070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 -- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0c0b1f80 msgr2=0x7f2e0c14c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0c0b1f80 0x7f2e0c14c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 -- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e0c06c1a0 msgr2=0x7f2e0c070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e0c06c1a0 0x7f2e0c070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 -- 172.21.15.46:0/2474538481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dfc0094d0 con 0x7f2e0c0adf50 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0adf50 0x7f2e0c149370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c400 tx=0x7f2e0000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e097fa640 1 -- 172.21.15.46:0/2474538481 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00016070 con 0x7f2e0c0adf50 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e097fa640 1 -- 172.21.15.46:0/2474538481 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e00007020 con 0x7f2e0c0adf50 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c14a740 con 0x7f2e0c0adf50 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e097fa640 1 -- 172.21.15.46:0/2474538481 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00007410 con 0x7f2e0c0adf50 2023-12-16T21:42:51.895 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c149020 con 0x7f2e0c0adf50 2023-12-16T21:42:51.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e097fa640 1 -- 172.21.15.46:0/2474538481 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2e00007570 con 0x7f2e0c0adf50 2023-12-16T21:42:51.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e097fa640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2df8066470 0x7f2df8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e097fa640 1 -- 172.21.15.46:0/2474538481 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e000836a0 con 0x7f2e0c0adf50 2023-12-16T21:42:51.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.887+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2df8066470 0x7f2df8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2df8066470 0x7f2df8068930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0af000 tx=0x7f2dfc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 Processor -- start 2023-12-16T21:42:51.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- start start 2023-12-16T21:42:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564991c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956499700 0x55d95649e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649e790 0x55d95649eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d956335f00 con 0x55d956498de0 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9562c5dd0 con 0x55d95649e790 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9562c3300 con 0x55d956499700 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51534/0 (socket says 172.21.15.46:51534) 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 -- 172.21.15.46:0/877323246 learned_addr learned my addr 172.21.15.46:0/877323246 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956499700 0x55d95649e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e1161c640 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649e790 0x55d95649eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 -- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956499700 msgr2=0x55d95649e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956499700 0x55d95649e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 -- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649e790 msgr2=0x55d95649eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649e790 0x55d95649eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 -- 172.21.15.46:0/877323246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d955fb6890 con 0x55d956498de0 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e1161c640 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649e790 0x55d95649eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564991c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc01c970 tx=0x7f2dfc0a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbb317d827038fc server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2df27fc640 1 -- 172.21.15.46:0/877323246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0b4af0 con 0x55d956498de0 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2df27fc640 1 -- 172.21.15.46:0/877323246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc0bc040 con 0x55d956498de0 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2df27fc640 1 -- 172.21.15.46:0/877323246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0b4e50 con 0x55d956498de0 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- 172.21.15.46:0/877323246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 msgr2=0x55d9564991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564991c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc01c970 tx=0x7f2dfc0a7000 comp rx=0 tx=0).stop 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- 172.21.15.46:0/877323246 shutdown_connections 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649e790 0x55d95649eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956499700 0x55d95649e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- 172.21.15.46:0/877323246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564991c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.901 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- 172.21.15.46:0/877323246 >> 172.21.15.46:0/877323246 conn(0x55d95649a000 msgr2=0x55d95649c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- 172.21.15.46:0/877323246 shutdown_connections 2023-12-16T21:42:51.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- 172.21.15.46:0/877323246 wait complete. 2023-12-16T21:42:51.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- start start 2023-12-16T21:42:51.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 Processor -- start 2023-12-16T21:42:51.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956498de0 0x55d956473510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d956499700 0x55d9564a4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d95649e790 0x55d9564a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9562bebf0 con 0x55d95649e790 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d956276c60 con 0x55d956499700 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d95627cb00 con 0x55d956498de0 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e161b0000 10 client.0 _ll_register_callbacks cb 0x55d956459710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e1161c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d95649e790 0x55d9564a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e1161c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d95649e790 0x55d9564a4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51548/0 (socket says 172.21.15.46:51548) 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e1161c640 1 -- 172.21.15.46:0/2448460376 learned_addr learned my addr 172.21.15.46:0/2448460376 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956498de0 0x55d956473510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.891+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d956499700 0x55d9564a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e1161c640 1 -- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956498de0 msgr2=0x55d956473510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e1161c640 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956498de0 0x55d956473510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e1161c640 1 -- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d956499700 msgr2=0x55d9564a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e1161c640 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d956499700 0x55d9564a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e1161c640 1 -- 172.21.15.46:0/2448460376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c0ad600 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e1161c640 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d95649e790 0x55d9564a4a90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400e7f0 tx=0x7f2e0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1426fc7d46798b1a server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 1 -- 172.21.15.46:0/2448460376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400cd60 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d95627c490 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 1 -- 172.21.15.46:0/2448460376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e040039a0 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 1 -- 172.21.15.46:0/2448460376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e04008a30 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 10 client.4536 fetch_fsmap learned FSMap version 6 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d95627f200 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 1 -- 172.21.15.46:0/2448460376 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d95627c490 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 1 -- 172.21.15.46:0/2448460376 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 6) v1 ==== 2144+0+0 (secure 0 0 0) 0x7f2e04003b60 con 0x55d95649e790 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 10 client.4536 fetch_fsmap finished waiting for FSMap version 6 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 10 client.4536 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 20 client.4536 populate_metadata read hostname 'smithi046' 2023-12-16T21:42:51.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956473a50 0x55d9564a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 4 client.4536 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 --> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9562bd540 con 0x55d956473a50 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956473a50 0x55d9564a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956473a50 0x55d9564a54a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ce976b9933e418c server_cookie=c0e7514921793076 in_seq=0 out_seq=0 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 10 client.4536 ms_handle_connect on v2:172.21.15.161:6834/926962350 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 1 -- 172.21.15.46:0/2448460376 <== mds.0 v2:172.21.15.161:6834/926962350 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9562bd540 con 0x55d956473a50 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 10 client.4536 handle_command_reply: tid=0 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 client.4536 shutdown 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d95649e790 msgr2=0x55d9564a4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d95649e790 0x55d9564a4a90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2e0400e7f0 tx=0x7f2e0400ecb0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 shutdown_connections 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d95649e790 0x55d9564a4a90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d956499700 0x55d9564a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956473a50 0x55d9564a54a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 --2- 172.21.15.46:0/2448460376 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d956498de0 0x55d956473510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 >> 172.21.15.46:0/2448460376 conn(0x55d95649a000 msgr2=0x55d956474060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2deb7fe640 0 client.4536 ms_handle_reset on v2:172.21.15.161:6834/926962350 2023-12-16T21:42:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 shutdown_connections 2023-12-16T21:42:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.895+0000 7f2e161b0000 1 -- 172.21.15.46:0/2448460376 wait complete. 2023-12-16T21:42:51.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.899+0000 7f2e161b0000 20 client.4536 trim_cache size 0 max 16384 2023-12-16T21:42:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 Processor -- start 2023-12-16T21:42:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 -- start start 2023-12-16T21:42:51.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564a4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d955f77940 con 0x55d956498de0 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9562bda80 con 0x55d95649a000 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9562c2bf0 con 0x55d95649f440 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.911+0000 7f2e1161c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51550/0 (socket says 172.21.15.46:51550) 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 -- 172.21.15.46:0/524865683 learned_addr learned my addr 172.21.15.46:0/524865683 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 -- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 msgr2=0x55d9564732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564732c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 -- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 msgr2=0x55d9564a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 -- 172.21.15.46:0/524865683 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d956265500 con 0x55d956498de0 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564720a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e04000f90 tx=0x7f2e04011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d629e9e55e9c9c server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2df2ffd640 1 -- 172.21.15.46:0/524865683 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400e450 con 0x55d956498de0 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2df2ffd640 1 -- 172.21.15.46:0/524865683 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e040039a0 con 0x55d956498de0 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2df2ffd640 1 -- 172.21.15.46:0/524865683 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0400cb60 con 0x55d956498de0 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/524865683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 msgr2=0x55d9564720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564720a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2e04000f90 tx=0x7f2e04011000 comp rx=0 tx=0).stop 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/524865683 shutdown_connections 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564720a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- 172.21.15.46:0/524865683 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/524865683 >> 172.21.15.46:0/524865683 conn(0x55d95646aa00 msgr2=0x55d9564a54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/524865683 shutdown_connections 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/524865683 wait complete. 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- start start 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 Processor -- start 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564d4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564d4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564d4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9562bc400 con 0x55d956498de0 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d956259910 con 0x55d95649a000 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9562ba560 con 0x55d95649f440 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 10 client.0 _ll_register_callbacks cb 0x55d956241510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564d4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564d4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51558/0 (socket says 172.21.15.46:51558) 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 -- 172.21.15.46:0/4152925568 learned_addr learned my addr 172.21.15.46:0/4152925568 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:51.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 -- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 msgr2=0x55d9564d4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e1161c640 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564d4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564d4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564d4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 -- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 msgr2=0x55d9564d4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564d4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 -- 172.21.15.46:0/4152925568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d95627f200 con 0x55d956498de0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e10e1b640 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564d4300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0ad040 tx=0x7f2dfc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95713abb23cb0fe3 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2deb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2deb7fe640 1 -- 172.21.15.46:0/4152925568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc0cf040 con 0x55d956498de0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2deb7fe640 1 -- 172.21.15.46:0/4152925568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2dfc0042a0 con 0x55d956498de0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d956245a50 con 0x55d956498de0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2deb7fe640 1 -- 172.21.15.46:0/4152925568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dfc0b8040 con 0x55d956498de0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 10 client.4542 fetch_fsmap learned FSMap version 6 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55d955fb6cc0 con 0x55d956498de0 2023-12-16T21:42:51.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2deb7fe640 1 -- 172.21.15.46:0/4152925568 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d956245a50 con 0x55d956498de0 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2deb7fe640 1 -- 172.21.15.46:0/4152925568 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 6) v1 ==== 2144+0+0 (secure 0 0 0) 0x7f2dfc0bf070 con 0x55d956498de0 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 10 client.4542 fetch_fsmap finished waiting for FSMap version 6 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 10 client.4542 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 20 client.4542 populate_metadata read hostname 'smithi046' 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956499700 0x55d95650be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 4 client.4542 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.915+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 --> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d9562ecf30 con 0x55d956499700 2023-12-16T21:42:51.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956499700 0x55d95650be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2deb7fe640 10 client.4542 ms_handle_connect on v2:172.21.15.161:6834/926962350 2023-12-16T21:42:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e0bfff640 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956499700 0x55d95650be90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2158115db735ac server_cookie=df389f46ffe2d4fb in_seq=0 out_seq=0 2023-12-16T21:42:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2deb7fe640 1 -- 172.21.15.46:0/4152925568 <== mds.0 v2:172.21.15.161:6834/926962350 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d9562ecf30 con 0x55d956499700 2023-12-16T21:42:51.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2deb7fe640 10 client.4542 handle_command_reply: tid=0 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 client.4542 shutdown 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 msgr2=0x55d9564d4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564d4300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0ad040 tx=0x7f2dfc0046d0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 shutdown_connections 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:6834/926962350,v1:172.21.15.161:6835/926962350] conn(0x55d956499700 0x55d95650be90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55d95649f440 0x55d9564d4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55d95649a000 0x55d9564d4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2deb7fe640 0 client.4542 ms_handle_reset on v2:172.21.15.161:6834/926962350 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 --2- 172.21.15.46:0/4152925568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d956498de0 0x55d9564d4300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 >> 172.21.15.46:0/4152925568 conn(0x55d95646aa00 msgr2=0x55d9564a5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 shutdown_connections 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 1 -- 172.21.15.46:0/4152925568 wait complete. 2023-12-16T21:42:51.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e161b0000 20 client.4542 trim_cache size 0 max 16384 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "52c183c6-9df9-436e-b409-f347731c6c1e", 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4288, 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 10.514727358, 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 6, 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 21, 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 21, 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 17.613812961000001 2023-12-16T21:42:51.932 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:42:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2df8066470 msgr2=0x7f2df8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2df8066470 0x7f2df8068930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0af000 tx=0x7f2dfc008040 comp rx=0 tx=0).stop 2023-12-16T21:42:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0adf50 msgr2=0x7f2e0c149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:51.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0adf50 0x7f2e0c149370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000c400 tx=0x7f2e0000c8c0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 shutdown_connections 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2df8066470 0x7f2df8068930 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2e0c0b1f80 0x7f2e0c14c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e0c0adf50 0x7f2e0c149370 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 --2- 172.21.15.46:0/2474538481 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2e0c06c1a0 0x7f2e0c070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 >> 172.21.15.46:0/2474538481 conn(0x7f2e0c06a580 msgr2=0x7f2e0c09c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 shutdown_connections 2023-12-16T21:42:51.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:51.919+0000 7f2e1307f640 1 -- 172.21.15.46:0/2474538481 wait complete. 2023-12-16T21:42:51.942 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-12-16T21:42:51.966 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-12-16T21:42:51.966 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-12-16T21:42:51.966 INFO:tasks.ceph_fuse:client.0 config is {} 2023-12-16T21:42:51.966 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-16T21:42:51.966 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:51.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:51.967 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-16T21:42:51.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:51.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-12-16T21:42:52.029 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2023-12-16T21:42:52.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:42:52.031 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-12-16T21:42:52.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-16T21:42:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.187+0000 7f3c288c4640 1 Processor -- start 2023-12-16T21:42:52.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.187+0000 7f3c288c4640 1 -- start start 2023-12-16T21:42:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c288c4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3c240b3fd0 0x7f3c2413e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c288c4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c240af2e0 con 0x7f3c2406c920 2023-12-16T21:42:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c288c4640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c240b1f50 con 0x7f3c240b0340 2023-12-16T21:42:52.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c288c4640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2406cd00 con 0x7f3c240b3fd0 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2259c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3c2406c920 0x7f3c240afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42068/0 (socket says 172.21.15.46:42068) 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2259c640 1 -- 172.21.15.46:0/4019447883 learned_addr learned my addr 172.21.15.46:0/4019447883 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/4019447883 >> v2:172.21.15.161:3300/0 conn(0x7f3c240b3fd0 0x7f3c2413e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3835693913 0 0) 0x7f3c240af2e0 con 0x7f3c2406c920 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c0c003530 con 0x7f3c2406c920 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 310188006 0 0) 0x7f3c0c003530 con 0x7f3c2406c920 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 >> v2:172.21.15.161:3300/0 conn(0x7f3c240b3fd0 msgr2=0x7f3c2413e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 --2- 172.21.15.46:0/4019447883 >> v2:172.21.15.161:3300/0 conn(0x7f3c240b3fd0 0x7f3c2413e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 >> v1:172.21.15.161:6789/0 conn(0x7f3c240b0340 legacy=0x7f3c2413e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240af530 con 0x7f3c2406c920 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f3c14001680 con 0x7f3c2406c920 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 >> v1:172.21.15.46:6789/0 conn(0x7f3c2406c920 legacy=0x7f3c240afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:52.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c003980 0x7f3c0c003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c0c004380 0x7f3c0c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c0c0050f0 0x7f3c0c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2406cd00 con 0x7f3c0c004380 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c00c030 con 0x7f3c0c0050f0 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0c00c1f0 con 0x7f3c0c003980 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c0c004380 0x7f3c0c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c2259c640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c003980 0x7f3c0c003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c22d9d640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c0c0050f0 0x7f3c0c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c21d9b640 1 -- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c003980 msgr2=0x7f3c0c003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:52.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c003980 0x7f3c0c003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c21d9b640 1 -- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c0c0050f0 msgr2=0x7f3c0c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c0c0050f0 0x7f3c0c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c22d9d640 1 -- 172.21.15.46:0/4019447883 reap_dead start 2023-12-16T21:42:52.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.191+0000 7f3c21d9b640 1 -- 172.21.15.46:0/4019447883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c003530 con 0x7f3c0c004380 2023-12-16T21:42:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c0c004380 0x7f3c0c004850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009ae0 tx=0x7f3c1801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a532026f57b637b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180a5640 con 0x7f3c0c004380 2023-12-16T21:42:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c180a57a0 con 0x7f3c0c004380 2023-12-16T21:42:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c2159a640 1 -- 172.21.15.46:0/4019447883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180a5b50 con 0x7f3c0c004380 2023-12-16T21:42:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- 172.21.15.46:0/4019447883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c0c004380 msgr2=0x7f3c0c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:52.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c0c004380 0x7f3c0c004850 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009ae0 tx=0x7f3c1801ca50 comp rx=0 tx=0).stop 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- 172.21.15.46:0/4019447883 shutdown_connections 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 --2- 172.21.15.46:0/4019447883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c0c004380 0x7f3c0c004850 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- 172.21.15.46:0/4019447883 >> 172.21.15.46:0/4019447883 conn(0x7f3c240b48e0 msgr2=0x7f3c240b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- 172.21.15.46:0/4019447883 shutdown_connections 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- 172.21.15.46:0/4019447883 wait complete. 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 Processor -- start 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- start start 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c004380 0x7f3c241381c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24138700 0x7f3c241df090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c241df5d0 0x7f3c241d5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c240ad000 con 0x7f3c24138700 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241dffb0 con 0x7f3c241df5d0 2023-12-16T21:42:52.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c288c4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c241397c0 con 0x7f3c0c004380 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24138700 0x7f3c241df090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24138700 0x7f3c241df090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51570/0 (socket says 172.21.15.46:51570) 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 -- 172.21.15.46:0/2987411676 learned_addr learned my addr 172.21.15.46:0/2987411676 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c2259c640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c004380 0x7f3c241381c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c22d9d640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c241df5d0 0x7f3c241d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 -- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c004380 msgr2=0x7f3c241381c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c004380 0x7f3c241381c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 -- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c241df5d0 msgr2=0x7f3c241d5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c241df5d0 0x7f3c241d5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.195+0000 7f3c21d9b640 1 -- 172.21.15.46:0/2987411676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c240b4750 con 0x7f3c24138700 2023-12-16T21:42:52.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c21d9b640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24138700 0x7f3c241df090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c1801c880 tx=0x7f3c180197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c12ffd640 1 -- 172.21.15.46:0/2987411676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18004020 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c12ffd640 1 -- 172.21.15.46:0/2987411676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c180041c0 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c241dfa90 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c12ffd640 1 -- 172.21.15.46:0/2987411676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c180b8660 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c24141b00 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c240679e0 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c12ffd640 1 -- 172.21.15.46:0/2987411676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f3c180b4070 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c12ffd640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3c04066470 0x7f3c04068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c12ffd640 1 -- 172.21.15.46:0/2987411676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c1811e460 con 0x7f3c24138700 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c2259c640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3c04066470 0x7f3c04068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:52.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.199+0000 7f3c2259c640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3c04066470 0x7f3c04068930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c140010f0 tx=0x7f3c14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:52.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.203+0000 7f3c12ffd640 1 -- 172.21.15.46:0/2987411676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f3c180eb970 con 0x7f3c24138700 2023-12-16T21:42:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.391+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c24139900 con 0x7f3c24138700 2023-12-16T21:42:52.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.391+0000 7f3c12ffd640 1 -- 172.21.15.46:0/2987411676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3c24139900 con 0x7f3c24138700 2023-12-16T21:42:52.401 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-16T21:42:52.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3c04066470 msgr2=0x7f3c04068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3c04066470 0x7f3c04068930 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3c140010f0 tx=0x7f3c14009040 comp rx=0 tx=0).stop 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24138700 msgr2=0x7f3c241df090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24138700 0x7f3c241df090 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3c1801c880 tx=0x7f3c180197f0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 shutdown_connections 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3c04066470 0x7f3c04068930 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3c241df5d0 0x7f3c241d5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c24138700 0x7f3c241df090 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 --2- 172.21.15.46:0/2987411676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3c0c004380 0x7f3c241381c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 >> 172.21.15.46:0/2987411676 conn(0x7f3c240b48e0 msgr2=0x7f3c2406b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 shutdown_connections 2023-12-16T21:42:52.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:52.395+0000 7f3c288c4640 1 -- 172.21.15.46:0/2987411676 wait complete. 2023-12-16T21:42:52.418 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-16T21:42:52.418 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-16T21:42:52.418 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-16T21:42:52.418 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-16T21:42:52.419 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:52.419 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-16T21:42:52.419 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-16T21:42:52.419 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-16T21:42:52.419 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-16T21:42:52.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.419 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-16T21:42:52.425 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-16T21:42:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-16T21:42:52.425 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-16T21:42:52.426 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42451sec preferred_lft 42451sec 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:9b:f9:01:fd brd ff:ff:ff:ff:ff:ff 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-16T21:42:52.427 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:42:52.428 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-16T21:42:52.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.428 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-16T21:42:52.428 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-16T21:42:52.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2023-12-16T21:42:52.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2023-12-16T21:42:52.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2023-12-16T21:42:52.428 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-16T21:42:52.428 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-16T21:42:52.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.503 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-16T21:42:52.552 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-16T21:42:52.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.553 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-16T21:42:52.620 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-16T21:42:52.620 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-16T21:42:52.620 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-16T21:42:52.620 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-16T21:42:52.620 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-16T21:42:52.620 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-16T21:42:52.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.621 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-16T21:42:52.621 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-16T21:42:52.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-16T21:42:52.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-16T21:42:52.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-16T21:42:52.621 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-16T21:42:52.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.698 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-16T21:42:52.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.746 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2023-12-16T21:42:52.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.794 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-16T21:42:52.794 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-16T21:42:52.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:42:52.795 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-16T21:42:52.795 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-16T21:42:52.862 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-16T21:42:52.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-16T21:42:52.863 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-16T21:42:52.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:52.972 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-16T21:42:52.973 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-16T21:42:52.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-16T21:42:52.973 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2023-12-16T21:42:52.973 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-16T21:42:52.991 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-16T21:42:52.991 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-16T21:42:52.991 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:53.041 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-16T21:42:53.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-16T21:42:53.041 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:53.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-16T21:42:53.145 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2023-12-16T21:42:53.195 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2023-12-16T21:42:53.196 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:42:53.197 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:42:53.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:53.198 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-16T21:42:53.245 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-16T21:42:53.245 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-16T21:42:53.290 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-16T21:42:53.304 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-16T21:42:53.353 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-16T21:42:53.354 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-16T21:42:53.355 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-16T21:42:53.356 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-16T21:42:53.356 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:42:53.356 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:42:53.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:53.357 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-16T21:42:53.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-16T21:42:53.371+0000 7f76526bd580 -1 init, newargv = 0x563ea3af1440 newargc=15 2023-12-16T21:42:53.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[44476]: starting ceph client 2023-12-16T21:42:53.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[44476]: starting fuse 2023-12-16T21:42:54.483 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-16T21:42:54.502 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-16T21:42:54.552 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-16T21:42:54.552 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-16T21:42:54.552 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-16T21:42:54.552 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-16T21:42:54.552 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-16T21:42:54.552 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-16T21:42:54.553 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:42:54.554 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:54.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:54.555 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-16T21:42:54.606 INFO:teuthology.orchestra.run.smithi046.stdout:45 2023-12-16T21:42:54.606 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-12-16T21:42:54.606 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-16T21:42:54.606 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-16T21:42:54.606 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-16T21:42:54.607 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-16T21:42:54.608 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-16T21:42:54.733 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.44476.asok 2023-12-16T21:42:54.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-16T21:42:54.746 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44476.asok 2023-12-16T21:42:54.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:54.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44476.asok status 2023-12-16T21:42:54.903 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:42:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-16T21:42:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "c7586e95519b7be76f3b6e3a4b333888baf46bc9", 2023-12-16T21:42:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-202-gc7586e95 (c7586e95519b7be76f3b6e3a4b333888baf46bc9) reef (dev)", 2023-12-16T21:42:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-16T21:42:54.903 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "44476", 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4557, 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 4557 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-16T21:42:54.904 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4163569901 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4163569901 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.4557 192.168.0.1:0/4163569901", 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/4163569901", 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 6, 2023-12-16T21:42:54.905 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 21, 2023-12-16T21:42:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-16T21:42:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-16T21:42:54.906 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-16T21:42:54.906 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:42:54.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:54.917 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-16T21:42:54.917 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:54.923 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-16T21:42:54.923 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:54.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:42:54.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-16T21:42:54.977 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-16T21:42:54.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2ebbef5640 1 Processor -- start 2023-12-16T21:42:55.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2ebbef5640 1 -- start start 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2ebbef5640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2eb406c920 0x7f2eb40afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2ebbef5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2eb40b3fd0 0x7f2eb40b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2ebbef5640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2eb40af2e0 con 0x7f2eb409dd50 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2ebbef5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb40b1f50 con 0x7f2eb40b3fd0 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2ebbef5640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2eb406cd00 con 0x7f2eb406c920 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9490640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2eb40b3fd0 0x7f2eb40b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9490640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2eb40b3fd0 0x7f2eb40b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51586/0 (socket says 172.21.15.46:51586) 2023-12-16T21:42:55.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9490640 1 -- 172.21.15.46:0/3307087492 learned_addr learned my addr 172.21.15.46:0/3307087492 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9c91640 1 --2- 172.21.15.46:0/3307087492 >> v2:172.21.15.161:3300/0 conn(0x7f2eb406c920 0x7f2eb40afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9490640 1 -- 172.21.15.46:0/3307087492 >> v2:172.21.15.161:3300/0 conn(0x7f2eb406c920 msgr2=0x7f2eb40afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9490640 1 --2- 172.21.15.46:0/3307087492 >> v2:172.21.15.161:3300/0 conn(0x7f2eb406c920 0x7f2eb40afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9490640 1 -- 172.21.15.46:0/3307087492 >> v1:172.21.15.161:6789/0 conn(0x7f2eb409dd50 legacy=0x7f2eb413e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.143+0000 7f2eb9490640 1 -- 172.21.15.46:0/3307087492 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb40af530 con 0x7f2eb40b3fd0 2023-12-16T21:42:55.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2eb9490640 1 --2- 172.21.15.46:0/3307087492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb40b3fd0 0x7f2eb40b0170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40b5920 tx=0x7f2ea401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0723e3ad6e6842 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2eb8c8f640 1 -- 172.21.15.46:0/3307087492 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40ab600 con 0x7f2eb40b3fd0 2023-12-16T21:42:55.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2eb8c8f640 1 -- 172.21.15.46:0/3307087492 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea40a7050 con 0x7f2eb40b3fd0 2023-12-16T21:42:55.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2eb8c8f640 1 -- 172.21.15.46:0/3307087492 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea40ab980 con 0x7f2eb40b3fd0 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 -- 172.21.15.46:0/3307087492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb40b3fd0 msgr2=0x7f2eb40b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/3307087492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb40b3fd0 0x7f2eb40b0170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eb40b5920 tx=0x7f2ea401c430 comp rx=0 tx=0).stop 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 -- 172.21.15.46:0/3307087492 shutdown_connections 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/3307087492 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb40b3fd0 0x7f2eb40b0170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/3307087492 >> v2:172.21.15.161:3300/0 conn(0x7f2eb406c920 0x7f2eb40afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 -- 172.21.15.46:0/3307087492 >> 172.21.15.46:0/3307087492 conn(0x7f2eb40b48e0 msgr2=0x7f2eb40b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 -- 172.21.15.46:0/3307087492 shutdown_connections 2023-12-16T21:42:55.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 -- 172.21.15.46:0/3307087492 wait complete. 2023-12-16T21:42:55.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 Processor -- start 2023-12-16T21:42:55.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.147+0000 7f2ebbef5640 1 -- start start 2023-12-16T21:42:55.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb406c920 0x7f2eb413cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2eb409dd50 0x7f2eb40ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2eb4142920 0x7f2eb40acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb413d9a0 con 0x7f2eb406c920 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb4144680 con 0x7f2eb4142920 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb4144960 con 0x7f2eb409dd50 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb406c920 0x7f2eb413cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb406c920 0x7f2eb413cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51600/0 (socket says 172.21.15.46:51600) 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 -- 172.21.15.46:0/1782710395 learned_addr learned my addr 172.21.15.46:0/1782710395 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9490640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2eb409dd50 0x7f2eb40ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eba492640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2eb4142920 0x7f2eb40acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 -- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2eb409dd50 msgr2=0x7f2eb40ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2eb409dd50 0x7f2eb40ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 -- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2eb4142920 msgr2=0x7f2eb40acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2eb4142920 0x7f2eb40acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 -- 172.21.15.46:0/1782710395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea40094d0 con 0x7f2eb406c920 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eba492640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2eb4142920 0x7f2eb40acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9c91640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb406c920 0x7f2eb413cad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c7f0 tx=0x7f2eb000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eaa7fc640 1 -- 172.21.15.46:0/1782710395 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0004220 con 0x7f2eb406c920 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eaa7fc640 1 -- 172.21.15.46:0/1782710395 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb00043a0 con 0x7f2eb406c920 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb40b4750 con 0x7f2eb406c920 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eaa7fc640 1 -- 172.21.15.46:0/1782710395 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb0004510 con 0x7f2eb406c920 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb413d6f0 con 0x7f2eb406c920 2023-12-16T21:42:55.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb40679e0 con 0x7f2eb406c920 2023-12-16T21:42:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eaa7fc640 1 -- 172.21.15.46:0/1782710395 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2eb0003a70 con 0x7f2eb406c920 2023-12-16T21:42:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eaa7fc640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2e98066470 0x7f2e98068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eaa7fc640 1 -- 172.21.15.46:0/1782710395 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2eb0083d80 con 0x7f2eb406c920 2023-12-16T21:42:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.151+0000 7f2eb9490640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2e98066470 0x7f2e98068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.155+0000 7f2eb9490640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2e98066470 0x7f2e98068930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40ad000 tx=0x7f2ea4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.155+0000 7f2eaa7fc640 1 -- 172.21.15.46:0/1782710395 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2eb0051290 con 0x7f2eb406c920 2023-12-16T21:42:55.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.311+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2eb40ae7d0 con 0x7f2eb406c920 2023-12-16T21:42:55.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.315+0000 7f2eaa7fc640 1 -- 172.21.15.46:0/1782710395 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f2eb001e090 con 0x7f2eb406c920 2023-12-16T21:42:55.323 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:55.323 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:55.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2e98066470 msgr2=0x7f2e98068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2e98066470 0x7f2e98068930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2ea40ad000 tx=0x7f2ea4008040 comp rx=0 tx=0).stop 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb406c920 msgr2=0x7f2eb413cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb406c920 0x7f2eb413cad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2eb000c7f0 tx=0x7f2eb000ccb0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 shutdown_connections 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2e98066470 0x7f2e98068930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2eb4142920 0x7f2eb40acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2eb409dd50 0x7f2eb40ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 --2- 172.21.15.46:0/1782710395 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2eb406c920 0x7f2eb413cad0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 >> 172.21.15.46:0/1782710395 conn(0x7f2eb40b48e0 msgr2=0x7f2eb40b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 shutdown_connections 2023-12-16T21:42:55.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.319+0000 7f2ebbef5640 1 -- 172.21.15.46:0/1782710395 wait complete. 2023-12-16T21:42:55.342 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:55.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:55.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff03a51b640 1 Processor -- start 2023-12-16T21:42:55.474 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff03a51b640 1 -- start start 2023-12-16T21:42:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff03a51b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 0x7ff0340aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff03a51b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff0340af300 0x7ff034144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff03a51b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0340b1730 con 0x7ff0340b0470 2023-12-16T21:42:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff03a51b640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03406cd70 con 0x7ff0340af300 2023-12-16T21:42:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff03a51b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340b0ad0 con 0x7ff0340b1eb0 2023-12-16T21:42:55.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff0340b0470 0x7ff0341479e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42074/0 (socket says 172.21.15.46:42074) 2023-12-16T21:42:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 -- 172.21.15.46:0/2016960223 learned_addr learned my addr 172.21.15.46:0/2016960223 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff0337fe640 1 --2- 172.21.15.46:0/2016960223 >> v2:172.21.15.161:3300/0 conn(0x7ff0340af300 0x7ff034144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff033fff640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 0x7ff0340aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659806454 0 0) 0x7ff0340b1730 con 0x7ff0340b0470 2023-12-16T21:42:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff020003530 con 0x7ff0340b0470 2023-12-16T21:42:55.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 401077170 0 0) 0x7ff020003530 con 0x7ff0340b0470 2023-12-16T21:42:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 msgr2=0x7ff0340aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 0x7ff0340aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 >> v2:172.21.15.161:3300/0 conn(0x7ff0340af300 msgr2=0x7ff034144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 --2- 172.21.15.46:0/2016960223 >> v2:172.21.15.161:3300/0 conn(0x7ff0340af300 0x7ff034144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03406c7f0 con 0x7ff0340b0470 2023-12-16T21:42:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7ff02c001680 con 0x7ff0340b0470 2023-12-16T21:42:55.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 >> v1:172.21.15.46:6789/0 conn(0x7ff0340b0470 legacy=0x7ff0341479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020003980 0x7ff020003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 0x7ff0200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff020004d10 0x7ff020005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03406cd70 con 0x7ff020004d10 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02000b990 con 0x7ff020003980 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02000bb50 con 0x7ff0340b1eb0 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff020004d10 0x7ff020005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff033fff640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020003980 0x7ff020003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff0337fe640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 0x7ff0200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 -- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 msgr2=0x7ff0200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff0340b1eb0 0x7ff0200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 -- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020003980 msgr2=0x7ff020003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020003980 0x7ff020003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff038ab8640 1 -- 172.21.15.46:0/2016960223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff020003530 con 0x7ff020004d10 2023-12-16T21:42:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.467+0000 7ff033fff640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff020003980 0x7ff020003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff038ab8640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff020004d10 0x7ff020005190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff0340b08a0 tx=0x7ff02c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95f39c63019c028 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0a58f0 con 0x7ff020004d10 2023-12-16T21:42:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c0a5a50 con 0x7ff020004d10 2023-12-16T21:42:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff032ffd640 1 -- 172.21.15.46:0/2016960223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0a5e00 con 0x7ff020004d10 2023-12-16T21:42:55.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 -- 172.21.15.46:0/2016960223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff020004d10 msgr2=0x7ff020005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 --2- 172.21.15.46:0/2016960223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff020004d10 0x7ff020005190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff0340b08a0 tx=0x7ff02c01cca0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff038ab8640 1 -- 172.21.15.46:0/2016960223 reap_dead start 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 -- 172.21.15.46:0/2016960223 shutdown_connections 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 -- 172.21.15.46:0/2016960223 >> 172.21.15.46:0/2016960223 conn(0x7ff0340b4900 msgr2=0x7ff0340b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 -- 172.21.15.46:0/2016960223 shutdown_connections 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 -- 172.21.15.46:0/2016960223 wait complete. 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 Processor -- start 2023-12-16T21:42:55.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.471+0000 7ff03a51b640 1 -- start start 2023-12-16T21:42:55.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff020004d10 0x7ff0341dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034140100 0x7ff034140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff03413bba0 0x7ff03413c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034140ac0 con 0x7ff034140100 2023-12-16T21:42:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0341e0440 con 0x7ff03413bba0 2023-12-16T21:42:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0341e0720 con 0x7ff020004d10 2023-12-16T21:42:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034140100 0x7ff034140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034140100 0x7ff034140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51610/0 (socket says 172.21.15.46:51610) 2023-12-16T21:42:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 -- 172.21.15.46:0/1285114021 learned_addr learned my addr 172.21.15.46:0/1285114021 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff033fff640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff020004d10 0x7ff0341dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff038ab8640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff03413bba0 0x7ff03413c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 -- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff020004d10 msgr2=0x7ff0341dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff020004d10 0x7ff0341dfd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 -- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff03413bba0 msgr2=0x7ff03413c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff03413bba0 0x7ff03413c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 -- 172.21.15.46:0/1285114021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03413c8d0 con 0x7ff034140100 2023-12-16T21:42:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff038ab8640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff03413bba0 0x7ff03413c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:55.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff0337fe640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034140100 0x7ff034140580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff024009f90 tx=0x7ff0240115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff030ff9640 1 -- 172.21.15.46:0/1285114021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02400e030 con 0x7ff034140100 2023-12-16T21:42:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff030ff9640 1 -- 172.21.15.46:0/1285114021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff024016470 con 0x7ff034140100 2023-12-16T21:42:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03413f180 con 0x7ff034140100 2023-12-16T21:42:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff030ff9640 1 -- 172.21.15.46:0/1285114021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024009b80 con 0x7ff034140100 2023-12-16T21:42:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0341410f0 con 0x7ff034140100 2023-12-16T21:42:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0341e08e0 con 0x7ff034140100 2023-12-16T21:42:55.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff030ff9640 1 -- 172.21.15.46:0/1285114021 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7ff024016820 con 0x7ff034140100 2023-12-16T21:42:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff030ff9640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff0140664a0 0x7ff014068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff030ff9640 1 -- 172.21.15.46:0/1285114021 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff02400f070 con 0x7ff034140100 2023-12-16T21:42:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.475+0000 7ff033fff640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff0140664a0 0x7ff014068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.479+0000 7ff033fff640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff0140664a0 0x7ff014068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0098d0 tx=0x7ff01c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.479+0000 7ff030ff9640 1 -- 172.21.15.46:0/1285114021 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7ff024052250 con 0x7ff034140100 2023-12-16T21:42:55.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff03413fe20 con 0x7ff034140100 2023-12-16T21:42:55.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff030ff9640 1 -- 172.21.15.46:0/1285114021 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff03413fe20 con 0x7ff034140100 2023-12-16T21:42:55.650 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:55.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff0140664a0 msgr2=0x7ff014068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff0140664a0 0x7ff014068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0098d0 tx=0x7ff01c008040 comp rx=0 tx=0).stop 2023-12-16T21:42:55.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034140100 msgr2=0x7ff034140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034140100 0x7ff034140580 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff024009f90 tx=0x7ff0240115a0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 shutdown_connections 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff0140664a0 0x7ff014068960 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff01c0098d0 tx=0x7ff01c008040 comp rx=0 tx=0).stop 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff03413bba0 0x7ff03413c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff034140100 0x7ff034140580 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 --2- 172.21.15.46:0/1285114021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff020004d10 0x7ff0341dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 >> 172.21.15.46:0/1285114021 conn(0x7ff0340b4900 msgr2=0x7ff034143680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 shutdown_connections 2023-12-16T21:42:55.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.643+0000 7ff03a51b640 1 -- 172.21.15.46:0/1285114021 wait complete. 2023-12-16T21:42:55.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:42:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83d1838640 1 Processor -- start 2023-12-16T21:42:55.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83d1838640 1 -- start start 2023-12-16T21:42:55.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83d1838640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 0x7f83cc149e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83d1838640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83cc0b4990 con 0x7f83cc14a350 2023-12-16T21:42:55.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83d1838640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83cc0b5de0 con 0x7f83cc0b6fc0 2023-12-16T21:42:55.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83d1838640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0b5b70 con 0x7f83cc0bc8a0 2023-12-16T21:42:55.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83c3fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f83cc14a350 0x7f83cc0b68b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42088/0 (socket says 172.21.15.46:42088) 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83c3fff640 1 -- 172.21.15.46:0/298543615 learned_addr learned my addr 172.21.15.46:0/298543615 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83caffd640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 0x7f83cc149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761281103 0 0) 0x7f83cc0b4990 con 0x7f83cc14a350 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83b8003530 con 0x7f83cc14a350 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271649206 0 0) 0x7f83b8003530 con 0x7f83cc14a350 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 msgr2=0x7f83cc149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 0x7f83cc149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 >> v1:172.21.15.161:6789/0 conn(0x7f83cc0b6fc0 legacy=0x7f83cc151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc14a7b0 con 0x7f83cc14a350 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f83b4001680 con 0x7f83cc14a350 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 >> v1:172.21.15.46:6789/0 conn(0x7f83cc14a350 legacy=0x7f83cc0b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83b8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 0x7f83b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8004d10 0x7f83b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc0b5b70 con 0x7f83b80039b0 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b800bb70 con 0x7f83b8004d10 2023-12-16T21:42:55.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b800bd30 con 0x7f83cc0bc8a0 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83caffd640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83b8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83c3fff640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 0x7f83b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83caffd640 1 -- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 msgr2=0x7f83b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83caffd640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc0bc8a0 0x7f83b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83caffd640 1 -- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8004d10 msgr2=0x7f83b8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83cb7fe640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8004d10 0x7f83b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83caffd640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8004d10 0x7f83b8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83caffd640 1 -- 172.21.15.46:0/298543615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8003530 con 0x7f83b80039b0 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.795+0000 7f83cb7fe640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83b8004d10 0x7f83b8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:55.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83b8003df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009ae0 tx=0x7f83bc01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c252a3ec75ce225 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0a5620 con 0x7f83b80039b0 2023-12-16T21:42:55.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc0a5780 con 0x7f83b80039b0 2023-12-16T21:42:55.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83ca7fc640 1 -- 172.21.15.46:0/298543615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc0a5b30 con 0x7f83b80039b0 2023-12-16T21:42:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- 172.21.15.46:0/298543615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 msgr2=0x7f83b8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 --2- 172.21.15.46:0/298543615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83b8003df0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009ae0 tx=0x7f83bc01ca50 comp rx=0 tx=0).stop 2023-12-16T21:42:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83cb7fe640 1 -- 172.21.15.46:0/298543615 reap_dead start 2023-12-16T21:42:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- 172.21.15.46:0/298543615 shutdown_connections 2023-12-16T21:42:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- 172.21.15.46:0/298543615 >> 172.21.15.46:0/298543615 conn(0x7f83cc0722b0 msgr2=0x7f83cc0bbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- 172.21.15.46:0/298543615 shutdown_connections 2023-12-16T21:42:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- 172.21.15.46:0/298543615 wait complete. 2023-12-16T21:42:55.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 Processor -- start 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- start start 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83cc13c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc1dfdf0 0x7f83cc145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc144780 0x7f83cc1451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc1df0c0 con 0x7f83b80039b0 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc1464f0 con 0x7f83cc144780 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83d1838640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83cc1467a0 con 0x7f83cc1dfdf0 2023-12-16T21:42:55.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83cc13c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83cc13c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51628/0 (socket says 172.21.15.46:51628) 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 -- 172.21.15.46:0/744652971 learned_addr learned my addr 172.21.15.46:0/744652971 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83cb7fe640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc144780 0x7f83cc1451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83c3fff640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc1dfdf0 0x7f83cc145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 -- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc1dfdf0 msgr2=0x7f83cc145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc1dfdf0 0x7f83cc145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 -- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc144780 msgr2=0x7f83cc1451c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc144780 0x7f83cc1451c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83caffd640 1 -- 172.21.15.46:0/744652971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83cc1df310 con 0x7f83b80039b0 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.799+0000 7f83c3fff640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc1dfdf0 0x7f83cc145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83caffd640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83cc13c740 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009ae0 tx=0x7f83b8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c8ff9640 1 -- 172.21.15.46:0/744652971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc004220 con 0x7f83b80039b0 2023-12-16T21:42:55.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c8ff9640 1 -- 172.21.15.46:0/744652971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc004380 con 0x7f83b80039b0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83cc1df5f0 con 0x7f83b80039b0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c8ff9640 1 -- 172.21.15.46:0/744652971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc003820 con 0x7f83b80039b0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83cc144c30 con 0x7f83b80039b0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83cc068960 con 0x7f83b80039b0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c8ff9640 1 -- 172.21.15.46:0/744652971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f83bc0b4040 con 0x7f83b80039b0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c8ff9640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f83ac066470 0x7f83ac068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c8ff9640 1 -- 172.21.15.46:0/744652971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83bc11e160 con 0x7f83b80039b0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c3fff640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f83ac066470 0x7f83ac068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:55.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.803+0000 7f83c3fff640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f83ac066470 0x7f83ac068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83b4001210 tx=0x7f83b4001dc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:55.820 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.807+0000 7f83c8ff9640 1 -- 172.21.15.46:0/744652971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f83bc0eb840 con 0x7f83b80039b0 2023-12-16T21:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.963+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f83cc1475a0 con 0x7f83b80039b0 2023-12-16T21:42:55.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.963+0000 7f83c8ff9640 1 -- 172.21.15.46:0/744652971 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f83bc0eee60 con 0x7f83b80039b0 2023-12-16T21:42:55.972 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f83ac066470 msgr2=0x7f83ac068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f83ac066470 0x7f83ac068930 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f83b4001210 tx=0x7f83b4001dc0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 msgr2=0x7f83cc13c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83cc13c740 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009ae0 tx=0x7f83b8005800 comp rx=0 tx=0).stop 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 shutdown_connections 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f83ac066470 0x7f83ac068930 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f83cc144780 0x7f83cc1451c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f83cc1dfdf0 0x7f83cc145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 --2- 172.21.15.46:0/744652971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b80039b0 0x7f83cc13c740 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:55.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 >> 172.21.15.46:0/744652971 conn(0x7f83cc0722b0 msgr2=0x7f83cc147f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:55.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 shutdown_connections 2023-12-16T21:42:55.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:55.967+0000 7f83d1838640 1 -- 172.21.15.46:0/744652971 wait complete. 2023-12-16T21:42:55.990 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:42:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab3375640 1 Processor -- start 2023-12-16T21:42:56.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab3375640 1 -- start start 2023-12-16T21:42:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab3375640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 0x7f7aac0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab3375640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac0b1730 con 0x7f7aac1451c0 2023-12-16T21:42:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab3375640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac06c7f0 con 0x7f7aac0af300 2023-12-16T21:42:56.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab3375640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac06cd70 con 0x7f7aac0b1eb0 2023-12-16T21:42:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1912640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7aac1451c0 0x7f7aac14ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42094/0 (socket says 172.21.15.46:42094) 2023-12-16T21:42:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1912640 1 -- 172.21.15.46:0/2896988200 learned_addr learned my addr 172.21.15.46:0/2896988200 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1111640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 0x7f7aac0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 956199714 0 0) 0x7f7aac0b1730 con 0x7f7aac1451c0 2023-12-16T21:42:56.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a98003530 con 0x7f7aac1451c0 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3785721322 0 0) 0x7f7a98003530 con 0x7f7aac1451c0 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 msgr2=0x7f7aac0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 0x7f7aac0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 >> v1:172.21.15.161:6789/0 conn(0x7f7aac0af300 legacy=0x7f7aac144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac0b2290 con 0x7f7aac1451c0 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f7aa8001680 con 0x7f7aac1451c0 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 >> v1:172.21.15.46:6789/0 conn(0x7f7aac1451c0 legacy=0x7f7aac14ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7a98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 0x7f7a980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a98004d10 0x7f7a98005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac06cd70 con 0x7f7a980039b0 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9800bba0 con 0x7f7a98004d10 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9800bd60 con 0x7f7aac0b1eb0 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1111640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7a98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab0910640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 0x7f7a980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1912640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a98004d10 0x7f7a98005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1111640 1 -- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 msgr2=0x7f7a980047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1111640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 0x7f7a980047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1111640 1 -- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a98004d10 msgr2=0x7f7a98005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1111640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a98004d10 0x7f7a98005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1111640 1 -- 172.21.15.46:0/2896988200 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a98003530 con 0x7f7a980039b0 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab1912640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7a98004d10 0x7f7a98005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:56.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.111+0000 7f7ab0910640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac0b1eb0 0x7f7a980047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:56.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab1111640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7a98003dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009fc0 tx=0x7f7aa401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a722dc0b26aff4cf server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40a5a60 con 0x7f7a980039b0 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa400f040 con 0x7f7a980039b0 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7aa3fff640 1 -- 172.21.15.46:0/2896988200 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40a5e00 con 0x7f7a980039b0 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- 172.21.15.46:0/2896988200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 msgr2=0x7f7a98003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 --2- 172.21.15.46:0/2896988200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7a98003dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009fc0 tx=0x7f7aa401cc60 comp rx=0 tx=0).stop 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab1912640 1 -- 172.21.15.46:0/2896988200 reap_dead start 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- 172.21.15.46:0/2896988200 shutdown_connections 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- 172.21.15.46:0/2896988200 >> 172.21.15.46:0/2896988200 conn(0x7f7aac0b4900 msgr2=0x7f7aac0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- 172.21.15.46:0/2896988200 shutdown_connections 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- 172.21.15.46:0/2896988200 wait complete. 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 Processor -- start 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- start start 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7aac14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aac1df140 0x7f7aac06f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac1df780 0x7f7aac1dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0aecb0 con 0x7f7a980039b0 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0712b0 con 0x7f7aac1df140 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab3375640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac070380 con 0x7f7aac1df780 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab1111640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7aac14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab1111640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7aac14f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51654/0 (socket says 172.21.15.46:51654) 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab1111640 1 -- 172.21.15.46:0/137693001 learned_addr learned my addr 172.21.15.46:0/137693001 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab0910640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aac1df140 0x7f7aac06f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.115+0000 7f7ab1912640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac1df780 0x7f7aac1dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab1111640 1 -- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac1df780 msgr2=0x7f7aac1dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab1111640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac1df780 0x7f7aac1dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab1111640 1 -- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aac1df140 msgr2=0x7f7aac06f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab1111640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aac1df140 0x7f7aac06f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab1111640 1 -- 172.21.15.46:0/137693001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac0ae940 con 0x7f7a980039b0 2023-12-16T21:42:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab1111640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7aac14f200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7aa401c6d0 tx=0x7f7aa4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7aa1ffb640 1 -- 172.21.15.46:0/137693001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4003ba0 con 0x7f7a980039b0 2023-12-16T21:42:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7aa1ffb640 1 -- 172.21.15.46:0/137693001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4003d00 con 0x7f7a980039b0 2023-12-16T21:42:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac14f740 con 0x7f7a980039b0 2023-12-16T21:42:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7aa1ffb640 1 -- 172.21.15.46:0/137693001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4004260 con 0x7f7a980039b0 2023-12-16T21:42:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac06fb60 con 0x7f7a980039b0 2023-12-16T21:42:56.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac0aead0 con 0x7f7a980039b0 2023-12-16T21:42:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7aa1ffb640 1 -- 172.21.15.46:0/137693001 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7aa40b4040 con 0x7f7a980039b0 2023-12-16T21:42:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7aa1ffb640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7a90066470 0x7f7a90068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7aa1ffb640 1 -- 172.21.15.46:0/137693001 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aa411e400 con 0x7f7a980039b0 2023-12-16T21:42:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab0910640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7a90066470 0x7f7a90068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.119+0000 7f7ab0910640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7a90066470 0x7f7a90068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7aac1df520 tx=0x7f7a9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.123+0000 7f7aa1ffb640 1 -- 172.21.15.46:0/137693001 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7aa40eb920 con 0x7f7a980039b0 2023-12-16T21:42:56.286 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.279+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7aac068960 con 0x7f7a980039b0 2023-12-16T21:42:56.287 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.279+0000 7f7aa1ffb640 1 -- 172.21.15.46:0/137693001 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7aa40eef40 con 0x7f7a980039b0 2023-12-16T21:42:56.287 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:42:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7a90066470 msgr2=0x7f7a90068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7a90066470 0x7f7a90068930 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7aac1df520 tx=0x7f7a9c009040 comp rx=0 tx=0).stop 2023-12-16T21:42:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 msgr2=0x7f7aac14f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7aac14f200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7aa401c6d0 tx=0x7f7aa4009750 comp rx=0 tx=0).stop 2023-12-16T21:42:56.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 shutdown_connections 2023-12-16T21:42:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7a90066470 0x7f7a90068930 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7aac1df780 0x7f7aac1dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7aac1df140 0x7f7aac06f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 --2- 172.21.15.46:0/137693001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a980039b0 0x7f7aac14f200 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 >> 172.21.15.46:0/137693001 conn(0x7f7aac0b4900 msgr2=0x7f7aac0b5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 shutdown_connections 2023-12-16T21:42:56.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.283+0000 7f7ab3375640 1 -- 172.21.15.46:0/137693001 wait complete. 2023-12-16T21:42:56.306 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:56.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.423+0000 7febdc7cd640 1 Processor -- start 2023-12-16T21:42:56.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.423+0000 7febdc7cd640 1 -- start start 2023-12-16T21:42:56.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febdc7cd640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7febd4071d70 0x7febd40b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febdc7cd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febd40af760 0x7febd40afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febdc7cd640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40b39f0 con 0x7febd413e410 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febdc7cd640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd4072150 con 0x7febd40af760 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febdc7cd640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40b55d0 con 0x7febd4071d70 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febd40af760 0x7febd40afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febd40af760 0x7febd40afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51656/0 (socket says 172.21.15.46:51656) 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 -- 172.21.15.46:0/4274823261 learned_addr learned my addr 172.21.15.46:0/4274823261 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febdad6a640 1 --1- 172.21.15.46:0/4274823261 >> v1:172.21.15.46:6789/0 conn(0x7febd413e410 0x7febd406a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42110/0 (socket says 172.21.15.46:42110) 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febda569640 1 --2- 172.21.15.46:0/4274823261 >> v2:172.21.15.161:3300/0 conn(0x7febd4071d70 0x7febd40b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 -- 172.21.15.46:0/4274823261 >> v2:172.21.15.161:3300/0 conn(0x7febd4071d70 msgr2=0x7febd40b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 --2- 172.21.15.46:0/4274823261 >> v2:172.21.15.161:3300/0 conn(0x7febd4071d70 0x7febd40b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 -- 172.21.15.46:0/4274823261 >> v1:172.21.15.46:6789/0 conn(0x7febd413e410 legacy=0x7febd406a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 -- 172.21.15.46:0/4274823261 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd40b1120 con 0x7febd40af760 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9567640 1 -- 172.21.15.46:0/4274823261 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3260509533 0 0) 0x7febd40b39f0 con 0x7febd413e410 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9d68640 1 --2- 172.21.15.46:0/4274823261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af760 0x7febd40afc10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febd406b5f0 tx=0x7febc801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab05f0bf7a00117 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9567640 1 -- 172.21.15.46:0/4274823261 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80ab600 con 0x7febd40af760 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9567640 1 -- 172.21.15.46:0/4274823261 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc80a7050 con 0x7febd40af760 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.427+0000 7febd9567640 1 -- 172.21.15.46:0/4274823261 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc80ab980 con 0x7febd40af760 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- 172.21.15.46:0/4274823261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af760 msgr2=0x7febd40afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 --2- 172.21.15.46:0/4274823261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af760 0x7febd40afc10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7febd406b5f0 tx=0x7febc801c3e0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- 172.21.15.46:0/4274823261 shutdown_connections 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 --2- 172.21.15.46:0/4274823261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af760 0x7febd40afc10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 --2- 172.21.15.46:0/4274823261 >> v2:172.21.15.161:3300/0 conn(0x7febd4071d70 0x7febd40b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- 172.21.15.46:0/4274823261 >> 172.21.15.46:0/4274823261 conn(0x7febd40b9250 msgr2=0x7febd40b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- 172.21.15.46:0/4274823261 shutdown_connections 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- 172.21.15.46:0/4274823261 wait complete. 2023-12-16T21:42:56.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 Processor -- start 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- start start 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd4071d70 0x7febd413d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40af760 0x7febd40adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413e410 0x7febd40ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40ad6b0 con 0x7febd413e410 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd40ae5c0 con 0x7febd40af760 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd40acdd0 con 0x7febd4071d70 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413e410 0x7febd40ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413e410 0x7febd40ae080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51668/0 (socket says 172.21.15.46:51668) 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 -- 172.21.15.46:0/403537205 learned_addr learned my addr 172.21.15.46:0/403537205 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febda569640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd4071d70 0x7febd413d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 -- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd4071d70 msgr2=0x7febd413d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd4071d70 0x7febd413d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 -- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40af760 msgr2=0x7febd40adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febd9d68640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40af760 0x7febd40adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40af760 0x7febd40adb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 -- 172.21.15.46:0/403537205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80094d0 con 0x7febd413e410 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdad6a640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413e410 0x7febd40ae080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7febcc00c3c0 tx=0x7febcc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febd2ffd640 1 -- 172.21.15.46:0/403537205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc0074e0 con 0x7febd413e410 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febd2ffd640 1 -- 172.21.15.46:0/403537205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc007640 con 0x7febd413e410 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd40af310 con 0x7febd413e410 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febd2ffd640 1 -- 172.21.15.46:0/403537205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc004a30 con 0x7febd413e410 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.431+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd40ad050 con 0x7febd413e410 2023-12-16T21:42:56.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.435+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb4005260 con 0x7febd413e410 2023-12-16T21:42:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.435+0000 7febd2ffd640 1 -- 172.21.15.46:0/403537205 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7febcc004b90 con 0x7febd413e410 2023-12-16T21:42:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.435+0000 7febd2ffd640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.435+0000 7febd2ffd640 1 -- 172.21.15.46:0/403537205 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febcc083b80 con 0x7febd413e410 2023-12-16T21:42:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.435+0000 7febda569640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.435+0000 7febda569640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7febc4009f30 tx=0x7febc400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.439+0000 7febd2ffd640 1 -- 172.21.15.46:0/403537205 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7febcc051090 con 0x7febd413e410 2023-12-16T21:42:56.606 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.599+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febb40057e0 con 0x7febd413e410 2023-12-16T21:42:56.609 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febd2ffd640 1 -- 172.21.15.46:0/403537205 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7febcc0546b0 con 0x7febd413e410 2023-12-16T21:42:56.609 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:56.610 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 msgr2=0x7febc0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7febc4009f30 tx=0x7febc400b040 comp rx=0 tx=0).stop 2023-12-16T21:42:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413e410 msgr2=0x7febd40ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413e410 0x7febd40ae080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7febcc00c3c0 tx=0x7febcc00c880 comp rx=0 tx=0).stop 2023-12-16T21:42:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 shutdown_connections 2023-12-16T21:42:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413e410 0x7febd40ae080 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40af760 0x7febd40adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 --2- 172.21.15.46:0/403537205 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd4071d70 0x7febd413d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 >> 172.21.15.46:0/403537205 conn(0x7febd40b9250 msgr2=0x7febd4144dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 shutdown_connections 2023-12-16T21:42:56.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.603+0000 7febdc7cd640 1 -- 172.21.15.46:0/403537205 wait complete. 2023-12-16T21:42:56.629 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:56.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:56.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820f46c640 1 Processor -- start 2023-12-16T21:42:56.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820f46c640 1 -- start start 2023-12-16T21:42:56.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820f46c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f820806b620 0x7f8208143830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820f46c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 0x7f8208151080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820f46c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8208073a90 con 0x7f82080736b0 2023-12-16T21:42:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820f46c640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82080af0e0 con 0x7f820806b620 2023-12-16T21:42:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820f46c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82080adb30 con 0x7f8208143d70 2023-12-16T21:42:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820d208640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f82080736b0 0x7f82080ad150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42114/0 (socket says 172.21.15.46:42114) 2023-12-16T21:42:56.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820d208640 1 -- 172.21.15.46:0/3551005416 learned_addr learned my addr 172.21.15.46:0/3551005416 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820da09640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 0x7f8208151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2352283207 0 0) 0x7f8208073a90 con 0x7f82080736b0 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f0003530 con 0x7f82080736b0 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820ca07640 1 --2- 172.21.15.46:0/3551005416 >> v2:172.21.15.161:3300/0 conn(0x7f820806b620 0x7f8208143830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3502262420 0 0) 0x7f81f0003530 con 0x7f82080736b0 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 msgr2=0x7f8208151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 0x7f8208151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 >> v2:172.21.15.161:3300/0 conn(0x7f820806b620 msgr2=0x7f8208143830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 --2- 172.21.15.46:0/3551005416 >> v2:172.21.15.161:3300/0 conn(0x7f820806b620 0x7f8208143830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080ade10 con 0x7f82080736b0 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f81f8001a00 con 0x7f82080736b0 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 >> v1:172.21.15.46:6789/0 conn(0x7f82080736b0 legacy=0x7f82080ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81f0003980 0x7f81f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 0x7f81f00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0004d10 0x7f81f0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080af0e0 con 0x7f81f0004d10 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080adb30 con 0x7f81f0003980 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f000b990 con 0x7f8208143d70 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820da09640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0004d10 0x7f81f0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820ca07640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 0x7f81f00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820d208640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81f0003980 0x7f81f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820da09640 1 -- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 msgr2=0x7f81f00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820da09640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8208143d70 0x7f81f00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820da09640 1 -- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81f0003980 msgr2=0x7f81f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:56.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820da09640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f81f0003980 0x7f81f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.751+0000 7f820da09640 1 -- 172.21.15.46:0/3551005416 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0003530 con 0x7f81f0004d10 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820da09640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0004d10 0x7f81f0005190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ae0 tx=0x7f820001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35802bf78d8d1638 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000a5530 con 0x7f81f0004d10 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82000a5690 con 0x7f81f0004d10 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f8207fff640 1 -- 172.21.15.46:0/3551005416 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000a5a40 con 0x7f81f0004d10 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 -- 172.21.15.46:0/3551005416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0004d10 msgr2=0x7f81f0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 --2- 172.21.15.46:0/3551005416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f81f0004d10 0x7f81f0005190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ae0 tx=0x7f820001ca50 comp rx=0 tx=0).stop 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820da09640 1 -- 172.21.15.46:0/3551005416 reap_dead start 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 -- 172.21.15.46:0/3551005416 shutdown_connections 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 -- 172.21.15.46:0/3551005416 >> 172.21.15.46:0/3551005416 conn(0x7f820806a580 msgr2=0x7f820806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 -- 172.21.15.46:0/3551005416 shutdown_connections 2023-12-16T21:42:56.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 -- 172.21.15.46:0/3551005416 wait complete. 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 Processor -- start 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.755+0000 7f820f46c640 1 -- start start 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81f0004d10 0x7f8208141af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b42b0 0x7f820813ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82080b4730 0x7f820813bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080af870 con 0x7f82080b4730 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080b1320 con 0x7f82080b42b0 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82080b2160 con 0x7f81f0004d10 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82080b4730 0x7f820813bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82080b4730 0x7f820813bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51688/0 (socket says 172.21.15.46:51688) 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 -- 172.21.15.46:0/2970620249 learned_addr learned my addr 172.21.15.46:0/2970620249 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:56.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820d208640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81f0004d10 0x7f8208141af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820ca07640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b42b0 0x7f820813ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 -- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81f0004d10 msgr2=0x7f8208141af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81f0004d10 0x7f8208141af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 -- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b42b0 msgr2=0x7f820813ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b42b0 0x7f820813ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 -- 172.21.15.46:0/2970620249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080b2440 con 0x7f82080b4730 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820ca07640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b42b0 0x7f820813ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820da09640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82080b4730 0x7f820813bfb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ae0 tx=0x7f81f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f8205ffb640 1 -- 172.21.15.46:0/2970620249 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82000a5530 con 0x7f82080b4730 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f8205ffb640 1 -- 172.21.15.46:0/2970620249 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8200000e90 con 0x7f82080b4730 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82080b1600 con 0x7f82080b4730 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f8205ffb640 1 -- 172.21.15.46:0/2970620249 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8200004480 con 0x7f82080b4730 2023-12-16T21:42:56.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82080b4cb0 con 0x7f82080b4730 2023-12-16T21:42:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82080b10f0 con 0x7f82080b4730 2023-12-16T21:42:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f8205ffb640 1 -- 172.21.15.46:0/2970620249 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f82000b4040 con 0x7f82080b4730 2023-12-16T21:42:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f8205ffb640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81f40664a0 0x7f81f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:56.769 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f8205ffb640 1 -- 172.21.15.46:0/2970620249 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f820011d6f0 con 0x7f82080b4730 2023-12-16T21:42:56.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820d208640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81f40664a0 0x7f81f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:56.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.759+0000 7f820d208640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81f40664a0 0x7f81f4068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81f8000f90 tx=0x7f81f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:56.771 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.763+0000 7f8205ffb640 1 -- 172.21.15.46:0/2970620249 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f82000eab50 con 0x7f82080b4730 2023-12-16T21:42:56.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.919+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82080b25d0 con 0x7f82080b4730 2023-12-16T21:42:56.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f8205ffb640 1 -- 172.21.15.46:0/2970620249 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82000ee170 con 0x7f82080b4730 2023-12-16T21:42:56.929 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:56.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81f40664a0 msgr2=0x7f81f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81f40664a0 0x7f81f4068960 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f81f8000f90 tx=0x7f81f8009040 comp rx=0 tx=0).stop 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82080b4730 msgr2=0x7f820813bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82080b4730 0x7f820813bfb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8200009ae0 tx=0x7f81f0005800 comp rx=0 tx=0).stop 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 shutdown_connections 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81f40664a0 0x7f81f4068960 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82080b4730 0x7f820813bfb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b42b0 0x7f820813ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 --2- 172.21.15.46:0/2970620249 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f81f0004d10 0x7f8208141af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 >> 172.21.15.46:0/2970620249 conn(0x7f820806a580 msgr2=0x7f8208066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 shutdown_connections 2023-12-16T21:42:56.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:56.923+0000 7f820f46c640 1 -- 172.21.15.46:0/2970620249 wait complete. 2023-12-16T21:42:56.948 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:42:57.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29697ca640 1 Processor -- start 2023-12-16T21:42:57.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29697ca640 1 -- start start 2023-12-16T21:42:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29697ca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f29640b1eb0 0x7f29640aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29697ca640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f29640af300 0x7f2964144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29697ca640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29640b1730 con 0x7f29640b0470 2023-12-16T21:42:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29697ca640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296406cd70 con 0x7f29640b1eb0 2023-12-16T21:42:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29697ca640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29640b0ad0 con 0x7f29640af300 2023-12-16T21:42:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29637fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f29640b0470 0x7f29641479e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42128/0 (socket says 172.21.15.46:42128) 2023-12-16T21:42:57.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29637fe640 1 -- 172.21.15.46:0/3966229581 learned_addr learned my addr 172.21.15.46:0/3966229581 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f2962ffd640 1 --2- 172.21.15.46:0/3966229581 >> v2:172.21.15.46:3300/0 conn(0x7f29640b1eb0 0x7f29640aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29627fc640 1 --2- 172.21.15.46:0/3966229581 >> v2:172.21.15.161:3300/0 conn(0x7f29640af300 0x7f2964144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f2961ffb640 1 -- 172.21.15.46:0/3966229581 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299139166 0 0) 0x7f29640b1730 con 0x7f29640b0470 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f2961ffb640 1 -- 172.21.15.46:0/3966229581 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c003530 con 0x7f29640b0470 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f2962ffd640 1 -- 172.21.15.46:0/3966229581 >> v2:172.21.15.161:3300/0 conn(0x7f29640af300 msgr2=0x7f2964144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f2962ffd640 1 --2- 172.21.15.46:0/3966229581 >> v2:172.21.15.161:3300/0 conn(0x7f29640af300 0x7f2964144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f2962ffd640 1 -- 172.21.15.46:0/3966229581 >> v1:172.21.15.46:6789/0 conn(0x7f29640b0470 legacy=0x7f29641479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f2962ffd640 1 -- 172.21.15.46:0/3966229581 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296406c7f0 con 0x7f29640b1eb0 2023-12-16T21:42:57.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.071+0000 7f29627fc640 1 --2- 172.21.15.46:0/3966229581 >> v2:172.21.15.161:3300/0 conn(0x7f29640af300 0x7f2964144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 --2- 172.21.15.46:0/3966229581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640b1eb0 0x7f29640aff30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f29640b59c0 tx=0x7f295401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22ca786baa5fa83 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2961ffb640 1 -- 172.21.15.46:0/3966229581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540a58b0 con 0x7f29640b1eb0 2023-12-16T21:42:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2961ffb640 1 -- 172.21.15.46:0/3966229581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540a5a10 con 0x7f29640b1eb0 2023-12-16T21:42:57.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2961ffb640 1 -- 172.21.15.46:0/3966229581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540a5da0 con 0x7f29640b1eb0 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- 172.21.15.46:0/3966229581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640b1eb0 msgr2=0x7f29640aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 --2- 172.21.15.46:0/3966229581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640b1eb0 0x7f29640aff30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f29640b59c0 tx=0x7f295401c7f0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- 172.21.15.46:0/3966229581 shutdown_connections 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 --2- 172.21.15.46:0/3966229581 >> v2:172.21.15.161:3300/0 conn(0x7f29640af300 0x7f2964144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 --2- 172.21.15.46:0/3966229581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640b1eb0 0x7f29640aff30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- 172.21.15.46:0/3966229581 >> 172.21.15.46:0/3966229581 conn(0x7f29640b4900 msgr2=0x7f29640b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- 172.21.15.46:0/3966229581 shutdown_connections 2023-12-16T21:42:57.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- 172.21.15.46:0/3966229581 wait complete. 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 Processor -- start 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- start start 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640af300 0x7f2964142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29640b0470 0x7f296413ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29640b1eb0 0x7f296413d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640ae0b0 con 0x7f29640af300 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29640ad5b0 con 0x7f29640b0470 2023-12-16T21:42:57.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296413d290 con 0x7f29640b1eb0 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640af300 0x7f2964142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640af300 0x7f2964142b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51712/0 (socket says 172.21.15.46:51712) 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 -- 172.21.15.46:0/2923351212 learned_addr learned my addr 172.21.15.46:0/2923351212 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29637fe640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29640b1eb0 0x7f296413d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29627fc640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29640b0470 0x7f296413ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 -- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29640b1eb0 msgr2=0x7f296413d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29640b1eb0 0x7f296413d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 -- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29640b0470 msgr2=0x7f296413ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29640b0470 0x7f296413ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 -- 172.21.15.46:0/2923351212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954009a70 con 0x7f29640af300 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29637fe640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29640b1eb0 0x7f296413d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29627fc640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29640b0470 0x7f296413ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f2962ffd640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640af300 0x7f2964142b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f295401a420 tx=0x7f2954019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f294b7fe640 1 -- 172.21.15.46:0/2923351212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540ad8d0 con 0x7f29640af300 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f294b7fe640 1 -- 172.21.15.46:0/2923351212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540ada50 con 0x7f29640af300 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296413cfb0 con 0x7f29640af300 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.075+0000 7f294b7fe640 1 -- 172.21.15.46:0/2923351212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540ade00 con 0x7f29640af300 2023-12-16T21:42:57.087 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.079+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2964146a30 con 0x7f29640af300 2023-12-16T21:42:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.079+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29640ade80 con 0x7f29640af300 2023-12-16T21:42:57.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.079+0000 7f294b7fe640 1 -- 172.21.15.46:0/2923351212 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f29540b9450 con 0x7f29640af300 2023-12-16T21:42:57.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.079+0000 7f294b7fe640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f29500664a0 0x7f2950068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.079+0000 7f294b7fe640 1 -- 172.21.15.46:0/2923351212 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f295411f2d0 con 0x7f29640af300 2023-12-16T21:42:57.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.079+0000 7f29627fc640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f29500664a0 0x7f2950068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.079+0000 7f29627fc640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f29500664a0 0x7f2950068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29641dd0d0 tx=0x7f2958008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.083+0000 7f294b7fe640 1 -- 172.21.15.46:0/2923351212 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f29640ade80 con 0x7f29640af300 2023-12-16T21:42:57.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.239+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29640ade80 con 0x7f29640af300 2023-12-16T21:42:57.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.239+0000 7f294b7fe640 1 -- 172.21.15.46:0/2923351212 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29640ade80 con 0x7f29640af300 2023-12-16T21:42:57.248 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f29500664a0 msgr2=0x7f2950068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f29500664a0 0x7f2950068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f29641dd0d0 tx=0x7f2958008040 comp rx=0 tx=0).stop 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640af300 msgr2=0x7f2964142b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640af300 0x7f2964142b20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f295401a420 tx=0x7f2954019590 comp rx=0 tx=0).stop 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 shutdown_connections 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f29500664a0 0x7f2950068960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29640b1eb0 0x7f296413d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29640b0470 0x7f296413ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 --2- 172.21.15.46:0/2923351212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29640af300 0x7f2964142b20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 >> 172.21.15.46:0/2923351212 conn(0x7f29640b4900 msgr2=0x7f29640b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:57.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 shutdown_connections 2023-12-16T21:42:57.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.243+0000 7f29697ca640 1 -- 172.21.15.46:0/2923351212 wait complete. 2023-12-16T21:42:57.268 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:42:57.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8737ce1640 1 Processor -- start 2023-12-16T21:42:57.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8737ce1640 1 -- start start 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8737ce1640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300af300 0x7f8730144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8737ce1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300b1730 con 0x7f87300b4350 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8737ce1640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873006c7f0 con 0x7f87300b1eb0 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8737ce1640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873006cd70 con 0x7f87300af300 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f873627e640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f87300b4350 0x7f873014ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42130/0 (socket says 172.21.15.46:42130) 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f873627e640 1 -- 172.21.15.46:0/1227781528 learned_addr learned my addr 172.21.15.46:0/1227781528 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f873527c640 1 --2- 172.21.15.46:0/1227781528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300af300 0x7f8730144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8734a7b640 1 -- 172.21.15.46:0/1227781528 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1138617455 0 0) 0x7f87300b1730 con 0x7f87300b4350 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8734a7b640 1 -- 172.21.15.46:0/1227781528 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f871c003530 con 0x7f87300b4350 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f8734a7b640 1 -- 172.21.15.46:0/1227781528 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041192906 0 0) 0x7f873006c7f0 con 0x7f87300b1eb0 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f873527c640 1 -- 172.21.15.46:0/1227781528 >> v1:172.21.15.161:6789/0 conn(0x7f87300b1eb0 legacy=0x7f87300aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f873527c640 1 -- 172.21.15.46:0/1227781528 >> v1:172.21.15.46:6789/0 conn(0x7f87300b4350 legacy=0x7f873014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.395+0000 7f873527c640 1 -- 172.21.15.46:0/1227781528 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300b2290 con 0x7f87300af300 2023-12-16T21:42:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f873527c640 1 --2- 172.21.15.46:0/1227781528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300af300 0x7f8730144ab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87200097f0 tx=0x7f872001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea3e0023f00238d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8734a7b640 1 -- 172.21.15.46:0/1227781528 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200ab600 con 0x7f87300af300 2023-12-16T21:42:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8734a7b640 1 -- 172.21.15.46:0/1227781528 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87200a7050 con 0x7f87300af300 2023-12-16T21:42:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8734a7b640 1 -- 172.21.15.46:0/1227781528 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200ab960 con 0x7f87300af300 2023-12-16T21:42:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- 172.21.15.46:0/1227781528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300af300 msgr2=0x7f8730144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 --2- 172.21.15.46:0/1227781528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300af300 0x7f8730144ab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f87200097f0 tx=0x7f872001c3a0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- 172.21.15.46:0/1227781528 shutdown_connections 2023-12-16T21:42:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 --2- 172.21.15.46:0/1227781528 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300af300 0x7f8730144ab0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- 172.21.15.46:0/1227781528 >> 172.21.15.46:0/1227781528 conn(0x7f87300b4900 msgr2=0x7f87300b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:57.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- 172.21.15.46:0/1227781528 shutdown_connections 2023-12-16T21:42:57.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- 172.21.15.46:0/1227781528 wait complete. 2023-12-16T21:42:57.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 Processor -- start 2023-12-16T21:42:57.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- start start 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87300af300 0x7f8730136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87300b1eb0 0x7f873006e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300b4350 0x7f873006ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730071030 con 0x7f87300af300 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8730071310 con 0x7f87300b1eb0 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8737ce1640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873013e8f0 con 0x7f87300b4350 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8735a7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87300af300 0x7f8730136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8735a7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87300af300 0x7f8730136eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51716/0 (socket says 172.21.15.46:51716) 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.399+0000 7f8735a7d640 1 -- 172.21.15.46:0/3255409711 learned_addr learned my addr 172.21.15.46:0/3255409711 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f873627e640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300b4350 0x7f873006ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f873527c640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87300b1eb0 0x7f873006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8735a7d640 1 -- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300b4350 msgr2=0x7f873006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8735a7d640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300b4350 0x7f873006ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8735a7d640 1 -- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87300b1eb0 msgr2=0x7f873006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8735a7d640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87300b1eb0 0x7f873006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8735a7d640 1 -- 172.21.15.46:0/3255409711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8720009420 con 0x7f87300af300 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f873527c640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87300b1eb0 0x7f873006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8735a7d640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87300af300 0x7f8730136eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f872c00c450 tx=0x7f872c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f87267fc640 1 -- 172.21.15.46:0/3255409711 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c005480 con 0x7f87300af300 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8730140240 con 0x7f87300af300 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f87267fc640 1 -- 172.21.15.46:0/3255409711 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c003600 con 0x7f87300af300 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f87267fc640 1 -- 172.21.15.46:0/3255409711 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c0039b0 con 0x7f87300af300 2023-12-16T21:42:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8730070070 con 0x7f87300af300 2023-12-16T21:42:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8730068960 con 0x7f87300af300 2023-12-16T21:42:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f87267fc640 1 -- 172.21.15.46:0/3255409711 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f872c003b10 con 0x7f87300af300 2023-12-16T21:42:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f87267fc640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f87140664a0 0x7f8714068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f87267fc640 1 -- 172.21.15.46:0/3255409711 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f872c083360 con 0x7f87300af300 2023-12-16T21:42:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f873527c640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f87140664a0 0x7f8714068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.403+0000 7f873527c640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f87140664a0 0x7f8714068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87200ad000 tx=0x7f87200a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.407+0000 7f87267fc640 1 -- 172.21.15.46:0/3255409711 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f872c0507f0 con 0x7f87300af300 2023-12-16T21:42:57.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f873013eab0 con 0x7f87300af300 2023-12-16T21:42:57.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f87267fc640 1 -- 172.21.15.46:0/3255409711 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f872c001130 con 0x7f87300af300 2023-12-16T21:42:57.573 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f87140664a0 msgr2=0x7f8714068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f87140664a0 0x7f8714068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f87200ad000 tx=0x7f87200a6000 comp rx=0 tx=0).stop 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87300af300 msgr2=0x7f8730136eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87300af300 0x7f8730136eb0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f872c00c450 tx=0x7f872c00c910 comp rx=0 tx=0).stop 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 shutdown_connections 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f87140664a0 0x7f8714068960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f87300b4350 0x7f873006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f87300b1eb0 0x7f873006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 --2- 172.21.15.46:0/3255409711 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f87300af300 0x7f8730136eb0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 >> 172.21.15.46:0/3255409711 conn(0x7f87300b4900 msgr2=0x7f87300b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 shutdown_connections 2023-12-16T21:42:57.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.567+0000 7f8737ce1640 1 -- 172.21.15.46:0/3255409711 wait complete. 2023-12-16T21:42:57.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:57.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.703+0000 7fd5e56cd640 1 Processor -- start 2023-12-16T21:42:57.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5e56cd640 1 -- start start 2023-12-16T21:42:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5e56cd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e006c920 0x7fd5e00ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5e56cd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5e00b4430 0x7fd5e00ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5e56cd640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e00af770 con 0x7fd5e00af2e0 2023-12-16T21:42:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5e56cd640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00afc20 con 0x7fd5e00b4430 2023-12-16T21:42:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5e56cd640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00aeea0 con 0x7fd5e006c920 2023-12-16T21:42:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5df7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd5e00b4430 0x7fd5e00ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5de7fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd5e00af2e0 0x7fd5e0147080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42138/0 (socket says 172.21.15.46:42138) 2023-12-16T21:42:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5de7fc640 1 -- 172.21.15.46:0/3721739476 learned_addr learned my addr 172.21.15.46:0/3721739476 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5deffd640 1 --2- 172.21.15.46:0/3721739476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e006c920 0x7fd5e00ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5df7fe640 1 -- 172.21.15.46:0/3721739476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e006c920 msgr2=0x7fd5e00ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5df7fe640 1 --2- 172.21.15.46:0/3721739476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e006c920 0x7fd5e00ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5df7fe640 1 -- 172.21.15.46:0/3721739476 >> v1:172.21.15.46:6789/0 conn(0x7fd5e00af2e0 legacy=0x7fd5e0147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5df7fe640 1 -- 172.21.15.46:0/3721739476 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e006cd00 con 0x7fd5e00b4430 2023-12-16T21:42:57.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.707+0000 7fd5ddffb640 1 -- 172.21.15.46:0/3721739476 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105022380 0 0) 0x7fd5e00af770 con 0x7fd5e00af2e0 2023-12-16T21:42:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5df7fe640 1 --2- 172.21.15.46:0/3721739476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e00b4430 0x7fd5e00ad5f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80097c0 tx=0x7fd5d801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abeff9a78024f3f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5ddffb640 1 -- 172.21.15.46:0/3721739476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80ab600 con 0x7fd5e00b4430 2023-12-16T21:42:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5ddffb640 1 -- 172.21.15.46:0/3721739476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d80a7050 con 0x7fd5e00b4430 2023-12-16T21:42:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5ddffb640 1 -- 172.21.15.46:0/3721739476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d80ab980 con 0x7fd5e00b4430 2023-12-16T21:42:57.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 -- 172.21.15.46:0/3721739476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e00b4430 msgr2=0x7fd5e00ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/3721739476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e00b4430 0x7fd5e00ad5f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80097c0 tx=0x7fd5d801c360 comp rx=0 tx=0).stop 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 -- 172.21.15.46:0/3721739476 shutdown_connections 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/3721739476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e00b4430 0x7fd5e00ad5f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/3721739476 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e006c920 0x7fd5e00ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 -- 172.21.15.46:0/3721739476 >> 172.21.15.46:0/3721739476 conn(0x7fd5e00b48e0 msgr2=0x7fd5e00b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 -- 172.21.15.46:0/3721739476 shutdown_connections 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 -- 172.21.15.46:0/3721739476 wait complete. 2023-12-16T21:42:57.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 Processor -- start 2023-12-16T21:42:57.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.711+0000 7fd5e56cd640 1 -- start start 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5e006c920 0x7fd5e0070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e00b4430 0x7fd5e0071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006f620 0x7fd5e006fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e006f190 con 0x7fd5e006f620 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e006fff0 con 0x7fd5e006c920 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0071770 con 0x7fd5e00b4430 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006f620 0x7fd5e006fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006f620 0x7fd5e006fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51724/0 (socket says 172.21.15.46:51724) 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 -- 172.21.15.46:0/1860865114 learned_addr learned my addr 172.21.15.46:0/1860865114 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:57.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5de7fc640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e00b4430 0x7fd5e0071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5deffd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5e006c920 0x7fd5e0070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 -- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e00b4430 msgr2=0x7fd5e0071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e00b4430 0x7fd5e0071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 -- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5e006c920 msgr2=0x7fd5e0070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5e006c920 0x7fd5e0070b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 -- 172.21.15.46:0/1860865114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d8009420 con 0x7fd5e006f620 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5de7fc640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e00b4430 0x7fd5e0071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5df7fe640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006f620 0x7fd5e006fab0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd5d801c840 tx=0x7fd5d8019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5c77fe640 1 -- 172.21.15.46:0/1860865114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8004230 con 0x7fd5e006f620 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5c77fe640 1 -- 172.21.15.46:0/1860865114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d8004390 con 0x7fd5e006f620 2023-12-16T21:42:57.723 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e013e8f0 con 0x7fd5e006f620 2023-12-16T21:42:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5c77fe640 1 -- 172.21.15.46:0/1860865114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8003820 con 0x7fd5e006f620 2023-12-16T21:42:57.724 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e013d910 con 0x7fd5e006f620 2023-12-16T21:42:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e00679e0 con 0x7fd5e006f620 2023-12-16T21:42:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5c77fe640 1 -- 172.21.15.46:0/1860865114 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fd5d80b2050 con 0x7fd5e006f620 2023-12-16T21:42:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5c77fe640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd5cc066470 0x7fd5cc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5c77fe640 1 -- 172.21.15.46:0/1860865114 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5d811e0d0 con 0x7fd5e006f620 2023-12-16T21:42:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.715+0000 7fd5deffd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd5cc066470 0x7fd5cc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:57.725 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.719+0000 7fd5deffd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd5cc066470 0x7fd5cc068930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0003fd0 tx=0x7fd5d00039c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:57.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.719+0000 7fd5c77fe640 1 -- 172.21.15.46:0/1860865114 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fd5d80eb5e0 con 0x7fd5e006f620 2023-12-16T21:42:57.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.879+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5e00688f0 con 0x7fd5e006f620 2023-12-16T21:42:57.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.879+0000 7fd5c77fe640 1 -- 172.21.15.46:0/1860865114 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fd5d80b0090 con 0x7fd5e006f620 2023-12-16T21:42:57.888 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:57.888 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd5cc066470 msgr2=0x7fd5cc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd5cc066470 0x7fd5cc068930 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0003fd0 tx=0x7fd5d00039c0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006f620 msgr2=0x7fd5e006fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006f620 0x7fd5e006fab0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd5d801c840 tx=0x7fd5d8019100 comp rx=0 tx=0).stop 2023-12-16T21:42:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 shutdown_connections 2023-12-16T21:42:57.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd5cc066470 0x7fd5cc068930 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd5e006f620 0x7fd5e006fab0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd5e00b4430 0x7fd5e0071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 --2- 172.21.15.46:0/1860865114 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd5e006c920 0x7fd5e0070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 >> 172.21.15.46:0/1860865114 conn(0x7fd5e00b48e0 msgr2=0x7fd5e00b0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 shutdown_connections 2023-12-16T21:42:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:57.883+0000 7fd5e56cd640 1 -- 172.21.15.46:0/1860865114 wait complete. 2023-12-16T21:42:57.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:57.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:58.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.023+0000 7f677e254640 1 Processor -- start 2023-12-16T21:42:58.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.023+0000 7f677e254640 1 -- start start 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f677e254640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780aded0 0x7f6778143160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f677e254640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f67781436a0 0x7f67780b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f677e254640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67780b3390 0x7f67781460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f677e254640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6778143b20 con 0x7f67780b3390 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f677e254640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67780ace20 con 0x7f67781436a0 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f677e254640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67780b3800 con 0x7f67780aded0 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67780b3390 0x7f67781460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f67780b3390 0x7f67781460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51730/0 (socket says 172.21.15.46:51730) 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 -- 172.21.15.46:0/1719722702 learned_addr learned my addr 172.21.15.46:0/1719722702 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f67777fe640 1 --2- 172.21.15.46:0/1719722702 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780aded0 0x7f6778143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6776ffd640 1 --2- 172.21.15.46:0/1719722702 >> v2:172.21.15.161:3300/0 conn(0x7f67781436a0 0x7f67780b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 -- 172.21.15.46:0/1719722702 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780aded0 msgr2=0x7f6778143160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 --2- 172.21.15.46:0/1719722702 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780aded0 0x7f6778143160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 -- 172.21.15.46:0/1719722702 >> v2:172.21.15.161:3300/0 conn(0x7f67781436a0 msgr2=0x7f67780b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 --2- 172.21.15.46:0/1719722702 >> v2:172.21.15.161:3300/0 conn(0x7f67781436a0 0x7f67780b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 -- 172.21.15.46:0/1719722702 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67780aeee0 con 0x7f67780b3390 2023-12-16T21:42:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f6777fff640 1 --2- 172.21.15.46:0/1719722702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781460a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f677806d020 tx=0x7f677001e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8d4331fe3a6038 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f67767fc640 1 -- 172.21.15.46:0/1719722702 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770004230 con 0x7f67780b3390 2023-12-16T21:42:58.036 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f67767fc640 1 -- 172.21.15.46:0/1719722702 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67700ab070 con 0x7f67780b3390 2023-12-16T21:42:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.027+0000 7f67767fc640 1 -- 172.21.15.46:0/1719722702 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6770003620 con 0x7f67780b3390 2023-12-16T21:42:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1719722702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 msgr2=0x7f67781460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.037 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 --2- 172.21.15.46:0/1719722702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781460a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f677806d020 tx=0x7f677001e890 comp rx=0 tx=0).stop 2023-12-16T21:42:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1719722702 shutdown_connections 2023-12-16T21:42:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 --2- 172.21.15.46:0/1719722702 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781460a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 --2- 172.21.15.46:0/1719722702 >> v2:172.21.15.161:3300/0 conn(0x7f67781436a0 0x7f67780b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 --2- 172.21.15.46:0/1719722702 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67780aded0 0x7f6778143160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1719722702 >> 172.21.15.46:0/1719722702 conn(0x7f67780b4900 msgr2=0x7f67780b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1719722702 shutdown_connections 2023-12-16T21:42:58.038 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1719722702 wait complete. 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 Processor -- start 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- start start 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780aded0 0x7f67780b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781deff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67781436a0 0x7f67781df530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677813d090 con 0x7f67780b3390 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677813d300 con 0x7f67780aded0 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677813be20 con 0x7f67781436a0 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781deff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781deff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51744/0 (socket says 172.21.15.46:51744) 2023-12-16T21:42:58.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 -- 172.21.15.46:0/1711955027 learned_addr learned my addr 172.21.15.46:0/1711955027 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f67777fe640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780aded0 0x7f67780b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6777fff640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67781436a0 0x7f67781df530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 -- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67781436a0 msgr2=0x7f67781df530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67781436a0 0x7f67781df530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 -- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780aded0 msgr2=0x7f67780b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780aded0 0x7f67780b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 -- 172.21.15.46:0/1711955027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677000b780 con 0x7f67780b3390 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f6776ffd640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781deff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f676c00d5f0 tx=0x7f676c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f675ffff640 1 -- 172.21.15.46:0/1711955027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0125f0 con 0x7f67780b3390 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f675ffff640 1 -- 172.21.15.46:0/1711955027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c012750 con 0x7f67780b3390 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f675ffff640 1 -- 172.21.15.46:0/1711955027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c012ae0 con 0x7f67780b3390 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f677813c100 con 0x7f67780b3390 2023-12-16T21:42:58.041 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67780b2290 con 0x7f67780b3390 2023-12-16T21:42:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.031+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f677813c340 con 0x7f67780b3390 2023-12-16T21:42:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.035+0000 7f675ffff640 1 -- 172.21.15.46:0/1711955027 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f676c012c40 con 0x7f67780b3390 2023-12-16T21:42:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.035+0000 7f675ffff640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f67640664a0 0x7f6764068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.042 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.035+0000 7f675ffff640 1 -- 172.21.15.46:0/1711955027 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f676c084c20 con 0x7f67780b3390 2023-12-16T21:42:58.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.035+0000 7f67777fe640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f67640664a0 0x7f6764068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.035+0000 7f67777fe640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f67640664a0 0x7f6764068960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6768003fd0 tx=0x7f67680039c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.039+0000 7f675ffff640 1 -- 172.21.15.46:0/1711955027 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f676c052210 con 0x7f67780b3390 2023-12-16T21:42:58.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67781dec50 con 0x7f67780b3390 2023-12-16T21:42:58.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f675ffff640 1 -- 172.21.15.46:0/1711955027 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f676c004650 con 0x7f67780b3390 2023-12-16T21:42:58.210 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f67640664a0 msgr2=0x7f6764068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f67640664a0 0x7f6764068960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6768003fd0 tx=0x7f67680039c0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 msgr2=0x7f67781deff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781deff0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f676c00d5f0 tx=0x7f676c00dab0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 shutdown_connections 2023-12-16T21:42:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f67640664a0 0x7f6764068960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f67781436a0 0x7f67781df530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f67780b3390 0x7f67781deff0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 --2- 172.21.15.46:0/1711955027 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f67780aded0 0x7f67780b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 >> 172.21.15.46:0/1711955027 conn(0x7f67780b4900 msgr2=0x7f67780b4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 shutdown_connections 2023-12-16T21:42:58.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.195+0000 7f677e254640 1 -- 172.21.15.46:0/1711955027 wait complete. 2023-12-16T21:42:58.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:42:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22da0b0640 1 Processor -- start 2023-12-16T21:42:58.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22da0b0640 1 -- start start 2023-12-16T21:42:58.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22da0b0640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 0x7f22d40aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.351 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22da0b0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22d40af300 0x7f22d4144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22da0b0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d40b1730 con 0x7f22d40b0470 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22da0b0640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d406cd70 con 0x7f22d40af300 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22da0b0640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d40b0ad0 con 0x7f22d40b1eb0 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d2ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f22d40af300 0x7f22d4144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d3fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f22d40b0470 0x7f22d41479e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42140/0 (socket says 172.21.15.46:42140) 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d3fff640 1 -- 172.21.15.46:0/3570105874 learned_addr learned my addr 172.21.15.46:0/3570105874 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d37fe640 1 --2- 172.21.15.46:0/3570105874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 0x7f22d40aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d2ffd640 1 -- 172.21.15.46:0/3570105874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 msgr2=0x7f22d40aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d2ffd640 1 --2- 172.21.15.46:0/3570105874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 0x7f22d40aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d2ffd640 1 -- 172.21.15.46:0/3570105874 >> v1:172.21.15.46:6789/0 conn(0x7f22d40b0470 legacy=0x7f22d41479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.343+0000 7f22d2ffd640 1 -- 172.21.15.46:0/3570105874 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d406c7f0 con 0x7f22d40af300 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d2ffd640 1 --2- 172.21.15.46:0/3570105874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d4144ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22c80097c0 tx=0x7f22c801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c8179905eac641 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d27fc640 1 -- 172.21.15.46:0/3570105874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80ad660 con 0x7f22d40af300 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d27fc640 1 -- 172.21.15.46:0/3570105874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c80ad7c0 con 0x7f22d40af300 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d27fc640 1 -- 172.21.15.46:0/3570105874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c80adb50 con 0x7f22d40af300 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- 172.21.15.46:0/3570105874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 msgr2=0x7f22d4144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 --2- 172.21.15.46:0/3570105874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d4144ab0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f22c80097c0 tx=0x7f22c801c360 comp rx=0 tx=0).stop 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- 172.21.15.46:0/3570105874 shutdown_connections 2023-12-16T21:42:58.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 --2- 172.21.15.46:0/3570105874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d4144ab0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 --2- 172.21.15.46:0/3570105874 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 0x7f22d40aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- 172.21.15.46:0/3570105874 >> 172.21.15.46:0/3570105874 conn(0x7f22d40b4900 msgr2=0x7f22d40b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- 172.21.15.46:0/3570105874 shutdown_connections 2023-12-16T21:42:58.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- 172.21.15.46:0/3570105874 wait complete. 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 Processor -- start 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- start start 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d413dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22d40b0470 0x7f22d41e0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 0x7f22d406fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4133130 con 0x7f22d40af300 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d406f630 con 0x7f22d40b0470 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4136c20 con 0x7f22d40b1eb0 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d413dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d413dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51766/0 (socket says 172.21.15.46:51766) 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 -- 172.21.15.46:0/1991436182 learned_addr learned my addr 172.21.15.46:0/1991436182 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 -- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 msgr2=0x7f22d406fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 0x7f22d406fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 -- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22d40b0470 msgr2=0x7f22d41e0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22d40b0470 0x7f22d41e0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 -- 172.21.15.46:0/1991436182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c8009420 con 0x7f22d40af300 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22d37fe640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d413dfb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d800 tx=0x7f22c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22bbfff640 1 -- 172.21.15.46:0/1991436182 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c400b6c0 con 0x7f22d40af300 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22bbfff640 1 -- 172.21.15.46:0/1991436182 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c400b820 con 0x7f22d40af300 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d41e11e0 con 0x7f22d40af300 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22bbfff640 1 -- 172.21.15.46:0/1991436182 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c400bb90 con 0x7f22d40af300 2023-12-16T21:42:58.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.347+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d4136940 con 0x7f22d40af300 2023-12-16T21:42:58.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.351+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d4068960 con 0x7f22d40af300 2023-12-16T21:42:58.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.351+0000 7f22bbfff640 1 -- 172.21.15.46:0/1991436182 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f22c400bcf0 con 0x7f22d40af300 2023-12-16T21:42:58.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.351+0000 7f22bbfff640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22c0066470 0x7f22c0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.351+0000 7f22bbfff640 1 -- 172.21.15.46:0/1991436182 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22c4084420 con 0x7f22d40af300 2023-12-16T21:42:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.351+0000 7f22d2ffd640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22c0066470 0x7f22c0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.351+0000 7f22d2ffd640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22c0066470 0x7f22c0068930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22c80af000 tx=0x7f22c8004740 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.362 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.355+0000 7f22bbfff640 1 -- 172.21.15.46:0/1991436182 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f22c4051890 con 0x7f22d40af300 2023-12-16T21:42:58.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.511+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f22d40b63f0 con 0x7f22d40af300 2023-12-16T21:42:58.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.511+0000 7f22bbfff640 1 -- 172.21.15.46:0/1991436182 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f22c4011340 con 0x7f22d40af300 2023-12-16T21:42:58.520 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:42:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22c0066470 msgr2=0x7f22c0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22c0066470 0x7f22c0068930 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f22c80af000 tx=0x7f22c8004740 comp rx=0 tx=0).stop 2023-12-16T21:42:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 msgr2=0x7f22d413dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d413dfb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f22c400d800 tx=0x7f22c400dcc0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 shutdown_connections 2023-12-16T21:42:58.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f22c0066470 0x7f22c0068930 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f22d40b1eb0 0x7f22d406fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f22d40b0470 0x7f22d41e0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 --2- 172.21.15.46:0/1991436182 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22d40af300 0x7f22d413dfb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 >> 172.21.15.46:0/1991436182 conn(0x7f22d40b4900 msgr2=0x7f22d413b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 shutdown_connections 2023-12-16T21:42:58.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.515+0000 7f22da0b0640 1 -- 172.21.15.46:0/1991436182 wait complete. 2023-12-16T21:42:58.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:42:58.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.647+0000 7f24cdf0f640 1 Processor -- start 2023-12-16T21:42:58.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.647+0000 7f24cdf0f640 1 -- start start 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24cdf0f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f24c80b3ff0 0x7f24c8147c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24cdf0f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c80b1f80 con 0x7f24c80b5f80 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24cdf0f640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c8148630 con 0x7f24c81481a0 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24cdf0f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c80b6360 con 0x7f24c80b3ff0 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f24c80b5f80 0x7f24c80b38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55108/0 (socket says 172.21.15.46:55108) 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 -- 172.21.15.46:0/3453267592 learned_addr learned my addr 172.21.15.46:0/3453267592 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824509500 0 0) 0x7f24c80b1f80 con 0x7f24c80b5f80 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c6ffd640 1 --2- 172.21.15.46:0/3453267592 >> v2:172.21.15.161:3300/0 conn(0x7f24c80b3ff0 0x7f24c8147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b0003530 con 0x7f24c80b5f80 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3679080535 0 0) 0x7f24b0003530 con 0x7f24c80b5f80 2023-12-16T21:42:58.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 >> v2:172.21.15.161:3300/0 conn(0x7f24c80b3ff0 msgr2=0x7f24c8147c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 --2- 172.21.15.46:0/3453267592 >> v2:172.21.15.161:3300/0 conn(0x7f24c80b3ff0 0x7f24c8147c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 >> v1:172.21.15.161:6789/0 conn(0x7f24c81481a0 legacy=0x7f24c814ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80b4760 con 0x7f24c80b5f80 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f24b8001680 con 0x7f24c80b5f80 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 >> v1:172.21.15.46:6789/0 conn(0x7f24c80b5f80 legacy=0x7f24c80b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b00047d0 0x7f24b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24b0005190 0x7f24b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80b6360 con 0x7f24b0003980 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b000c0e0 con 0x7f24b00047d0 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b000c2a0 con 0x7f24b0005190 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c6ffd640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b00047d0 0x7f24b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 -- 172.21.15.46:0/3453267592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24b0005190 msgr2=0x7f24b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:58.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24b0005190 0x7f24b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 -- 172.21.15.46:0/3453267592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b00047d0 msgr2=0x7f24b0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24b00047d0 0x7f24b0004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 -- 172.21.15.46:0/3453267592 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0003530 con 0x7f24b0003980 2023-12-16T21:42:58.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c7fff640 1 -- 172.21.15.46:0/3453267592 reap_dead start 2023-12-16T21:42:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c77fe640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24b0003da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c1c0 tx=0x7f24b801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3393d4718e723583 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8005ae0 con 0x7f24b0003980 2023-12-16T21:42:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8005c40 con 0x7f24b0003980 2023-12-16T21:42:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.651+0000 7f24c67fc640 1 -- 172.21.15.46:0/3453267592 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80072b0 con 0x7f24b0003980 2023-12-16T21:42:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/3453267592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 msgr2=0x7f24b0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24b0003da0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c1c0 tx=0x7f24b801cc20 comp rx=0 tx=0).stop 2023-12-16T21:42:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/3453267592 shutdown_connections 2023-12-16T21:42:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/3453267592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24b0003da0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/3453267592 >> 172.21.15.46:0/3453267592 conn(0x7f24c806a980 msgr2=0x7f24c806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/3453267592 shutdown_connections 2023-12-16T21:42:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/3453267592 wait complete. 2023-12-16T21:42:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 Processor -- start 2023-12-16T21:42:58.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- start start 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24c8143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c80b0210 0x7f24c806c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c806c670 0x7f24c80b7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c813d890 con 0x7f24b0003980 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80b9930 con 0x7f24c80b0210 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c806caf0 con 0x7f24c806c670 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24c8143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24c8143720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49378/0 (socket says 172.21.15.46:49378) 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 -- 172.21.15.46:0/455289264 learned_addr learned my addr 172.21.15.46:0/455289264 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c7fff640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c806c670 0x7f24c80b7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c6ffd640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c80b0210 0x7f24c806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 -- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c806c670 msgr2=0x7f24c80b7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c806c670 0x7f24c80b7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 -- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c80b0210 msgr2=0x7f24c806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c80b0210 0x7f24c806c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 -- 172.21.15.46:0/455289264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c81443b0 con 0x7f24b0003980 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24c77fe640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24c8143720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f24b801ca50 tx=0x7f24b0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24affff640 1 -- 172.21.15.46:0/455289264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80b4070 con 0x7f24b0003980 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24affff640 1 -- 172.21.15.46:0/455289264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8001540 con 0x7f24b0003980 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c80b92e0 con 0x7f24b0003980 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24affff640 1 -- 172.21.15.46:0/455289264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8007290 con 0x7f24b0003980 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c806ba80 con 0x7f24b0003980 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.655+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c80b1790 con 0x7f24b0003980 2023-12-16T21:42:58.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.659+0000 7f24affff640 1 -- 172.21.15.46:0/455289264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f24b80073f0 con 0x7f24b0003980 2023-12-16T21:42:58.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.659+0000 7f24affff640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f24b4066470 0x7f24b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.659+0000 7f24affff640 1 -- 172.21.15.46:0/455289264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24b811da10 con 0x7f24b0003980 2023-12-16T21:42:58.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.659+0000 7f24c6ffd640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f24b4066470 0x7f24b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.659+0000 7f24c6ffd640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f24b4066470 0x7f24b4068930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f24c813a390 tx=0x7f24bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.663+0000 7f24affff640 1 -- 172.21.15.46:0/455289264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f24b80eaf20 con 0x7f24b0003980 2023-12-16T21:42:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f24c80679e0 con 0x7f24b0003980 2023-12-16T21:42:58.822 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24affff640 1 -- 172.21.15.46:0/455289264 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24b80ee540 con 0x7f24b0003980 2023-12-16T21:42:58.822 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:42:58.824 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f24b4066470 msgr2=0x7f24b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f24b4066470 0x7f24b4068930 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f24c813a390 tx=0x7f24bc008040 comp rx=0 tx=0).stop 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 msgr2=0x7f24c8143720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24c8143720 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f24b801ca50 tx=0x7f24b0005800 comp rx=0 tx=0).stop 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 shutdown_connections 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f24b4066470 0x7f24b4068930 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f24c806c670 0x7f24c80b7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f24c80b0210 0x7f24c806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 --2- 172.21.15.46:0/455289264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f24b0003980 0x7f24c8143720 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.815+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 >> 172.21.15.46:0/455289264 conn(0x7f24c806a980 msgr2=0x7f24c8144ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.819+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 shutdown_connections 2023-12-16T21:42:58.825 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.819+0000 7f24cdf0f640 1 -- 172.21.15.46:0/455289264 wait complete. 2023-12-16T21:42:58.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:42:58.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.955+0000 7fb28f618640 1 Processor -- start 2023-12-16T21:42:58.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.955+0000 7fb28f618640 1 -- start start 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.955+0000 7fb28f618640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2880bc800 0x7fb2880b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.955+0000 7fb28f618640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb28813ff50 0x7fb2881403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.955+0000 7fb28f618640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2880bcbe0 con 0x7fb288149c80 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.955+0000 7fb28f618640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2880b38f0 con 0x7fb28813ff50 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.955+0000 7fb28f618640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2880b3bd0 con 0x7fb2880bc800 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2880bc800 0x7fb2880b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2880bc800 0x7fb2880b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:33736/0 (socket says 172.21.15.46:33736) 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28dbb5640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb28813ff50 0x7fb2881403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.965 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 -- 172.21.15.46:0/1149220069 learned_addr learned my addr 172.21.15.46:0/1149220069 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:58.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 -- 172.21.15.46:0/1149220069 >> v2:172.21.15.161:3300/0 conn(0x7fb28813ff50 msgr2=0x7fb2881403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:58.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 --2- 172.21.15.46:0/1149220069 >> v2:172.21.15.161:3300/0 conn(0x7fb28813ff50 0x7fb2881403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 -- 172.21.15.46:0/1149220069 >> v1:172.21.15.161:6789/0 conn(0x7fb288149c80 legacy=0x7fb28814a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 -- 172.21.15.46:0/1149220069 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880b3eb0 con 0x7fb2880bc800 2023-12-16T21:42:58.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb28d3b4640 1 --2- 172.21.15.46:0/1149220069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2880bc800 0x7fb2880b4bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb284009870 tx=0x7fb28401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3310ae344f8e40 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb27ffff640 1 -- 172.21.15.46:0/1149220069 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840ab650 con 0x7fb2880bc800 2023-12-16T21:42:58.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb27ffff640 1 -- 172.21.15.46:0/1149220069 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2840ab7b0 con 0x7fb2880bc800 2023-12-16T21:42:58.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.959+0000 7fb27ffff640 1 -- 172.21.15.46:0/1149220069 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2840abb40 con 0x7fb2880bc800 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/1149220069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2880bc800 msgr2=0x7fb2880b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 --2- 172.21.15.46:0/1149220069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2880bc800 0x7fb2880b4bb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb284009870 tx=0x7fb28401c540 comp rx=0 tx=0).stop 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/1149220069 shutdown_connections 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 --2- 172.21.15.46:0/1149220069 >> v2:172.21.15.161:3300/0 conn(0x7fb28813ff50 0x7fb2881403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 --2- 172.21.15.46:0/1149220069 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb2880bc800 0x7fb2880b4bb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/1149220069 >> 172.21.15.46:0/1149220069 conn(0x7fb2880722b0 msgr2=0x7fb2880bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/1149220069 shutdown_connections 2023-12-16T21:42:58.969 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/1149220069 wait complete. 2023-12-16T21:42:58.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 Processor -- start 2023-12-16T21:42:58.971 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- start start 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2880bc800 0x7fb28813d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb28813ff50 0x7fb28813d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288149c80 0x7fb2881e2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28813db30 con 0x7fb288149c80 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28814ad50 con 0x7fb2880bc800 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28814d900 con 0x7fb28813ff50 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288149c80 0x7fb2881e2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288149c80 0x7fb2881e2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49380/0 (socket says 172.21.15.46:49380) 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 -- 172.21.15.46:0/2144332596 learned_addr learned my addr 172.21.15.46:0/2144332596 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28cbb3640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb28813ff50 0x7fb28813d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28d3b4640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2880bc800 0x7fb28813d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 -- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb28813ff50 msgr2=0x7fb28813d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb28813ff50 0x7fb28813d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 -- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2880bc800 msgr2=0x7fb28813d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2880bc800 0x7fb28813d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 -- 172.21.15.46:0/2144332596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840094d0 con 0x7fb288149c80 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28dbb5640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288149c80 0x7fb2881e2370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb28000efc0 tx=0x7fb28000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb27dffb640 1 -- 172.21.15.46:0/2144332596 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280014480 con 0x7fb288149c80 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb27dffb640 1 -- 172.21.15.46:0/2144332596 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28000f040 con 0x7fb288149c80 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28814dbe0 con 0x7fb288149c80 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb27dffb640 1 -- 172.21.15.46:0/2144332596 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280014820 con 0x7fb288149c80 2023-12-16T21:42:58.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb288143950 con 0x7fb288149c80 2023-12-16T21:42:58.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.963+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb288067de0 con 0x7fb288149c80 2023-12-16T21:42:58.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.967+0000 7fb27dffb640 1 -- 172.21.15.46:0/2144332596 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fb280014980 con 0x7fb288149c80 2023-12-16T21:42:58.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.967+0000 7fb27dffb640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb26c066470 0x7fb26c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:58.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.967+0000 7fb27dffb640 1 -- 172.21.15.46:0/2144332596 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2800851d0 con 0x7fb288149c80 2023-12-16T21:42:58.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.967+0000 7fb28d3b4640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb26c066470 0x7fb26c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:58.974 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.967+0000 7fb28d3b4640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb26c066470 0x7fb26c068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2840af000 tx=0x7fb2840a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:58.977 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:58.971+0000 7fb27dffb640 1 -- 172.21.15.46:0/2144332596 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fb2800527c0 con 0x7fb288149c80 2023-12-16T21:42:59.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.131+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2880bbc50 con 0x7fb288149c80 2023-12-16T21:42:59.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.131+0000 7fb27dffb640 1 -- 172.21.15.46:0/2144332596 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fb280055de0 con 0x7fb288149c80 2023-12-16T21:42:59.140 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:42:59.140 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb26c066470 msgr2=0x7fb26c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb26c066470 0x7fb26c068930 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb2840af000 tx=0x7fb2840a7000 comp rx=0 tx=0).stop 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288149c80 msgr2=0x7fb2881e2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288149c80 0x7fb2881e2370 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb28000efc0 tx=0x7fb28000c490 comp rx=0 tx=0).stop 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 shutdown_connections 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb26c066470 0x7fb26c068930 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb288149c80 0x7fb2881e2370 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb28813ff50 0x7fb28813d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 --2- 172.21.15.46:0/2144332596 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb2880bc800 0x7fb28813d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 >> 172.21.15.46:0/2144332596 conn(0x7fb2880722b0 msgr2=0x7fb288145df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 shutdown_connections 2023-12-16T21:42:59.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.135+0000 7fb28f618640 1 -- 172.21.15.46:0/2144332596 wait complete. 2023-12-16T21:42:59.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:42:59.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:42:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad911b0640 1 Processor -- start 2023-12-16T21:42:59.289 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad911b0640 1 -- start start 2023-12-16T21:42:59.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad911b0640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fad8c140bb0 0x7fad8c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad911b0640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c0b4860 con 0x7fad8c0b6fc0 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad911b0640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c0b73a0 con 0x7fad8c0b5d80 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad911b0640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad8c06b8e0 con 0x7fad8c140bb0 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8ad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fad8c0b6fc0 0x7fad8c1404a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55112/0 (socket says 172.21.15.46:55112) 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8ad9d640 1 -- 172.21.15.46:0/1401433838 learned_addr learned my addr 172.21.15.46:0/1401433838 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198629546 0 0) 0x7fad8c0b4860 con 0x7fad8c0b6fc0 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad74003530 con 0x7fad8c0b6fc0 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8a59c640 1 --2- 172.21.15.46:0/1401433838 >> v2:172.21.15.161:3300/0 conn(0x7fad8c140bb0 0x7fad8c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248449153 0 0) 0x7fad74003530 con 0x7fad8c0b6fc0 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 >> v2:172.21.15.161:3300/0 conn(0x7fad8c140bb0 msgr2=0x7fad8c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 --2- 172.21.15.46:0/1401433838 >> v2:172.21.15.161:3300/0 conn(0x7fad8c140bb0 0x7fad8c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 >> v1:172.21.15.161:6789/0 conn(0x7fad8c0b5d80 legacy=0x7fad8c150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0b6260 con 0x7fad8c0b6fc0 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fad7c001a00 con 0x7fad8c0b6fc0 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 >> v1:172.21.15.46:6789/0 conn(0x7fad8c0b6fc0 legacy=0x7fad8c1404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad74003980 0x7fad74003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad740047d0 0x7fad74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad740052b0 0x7fad7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c06b8e0 con 0x7fad740052b0 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7400c180 con 0x7fad74003980 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7400c340 con 0x7fad740047d0 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8b59e640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad740052b0 0x7fad7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8ad9d640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad74003980 0x7fad74003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8a59c640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad740047d0 0x7fad74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8b59e640 1 -- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad740047d0 msgr2=0x7fad74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8b59e640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad740047d0 0x7fad74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8b59e640 1 -- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad74003980 msgr2=0x7fad74003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:59.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8b59e640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad74003980 0x7fad74003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.294 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.283+0000 7fad8b59e640 1 -- 172.21.15.46:0/1401433838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad74003530 con 0x7fad740052b0 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad8b59e640 1 -- 172.21.15.46:0/1401433838 reap_dead start 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad8b59e640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad740052b0 0x7fad7400b990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0b6190 tx=0x7fad8401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6ea594b4b98b27 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad840052d0 con 0x7fad740052b0 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad840a7050 con 0x7fad740052b0 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad89d9b640 1 -- 172.21.15.46:0/1401433838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84003600 con 0x7fad740052b0 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- 172.21.15.46:0/1401433838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad740052b0 msgr2=0x7fad7400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad740052b0 0x7fad7400b990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fad8c0b6190 tx=0x7fad8401c3a0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- 172.21.15.46:0/1401433838 shutdown_connections 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 --2- 172.21.15.46:0/1401433838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad740052b0 0x7fad7400b990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- 172.21.15.46:0/1401433838 >> 172.21.15.46:0/1401433838 conn(0x7fad8c06a980 msgr2=0x7fad8c06ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- 172.21.15.46:0/1401433838 shutdown_connections 2023-12-16T21:42:59.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- 172.21.15.46:0/1401433838 wait complete. 2023-12-16T21:42:59.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 Processor -- start 2023-12-16T21:42:59.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- start start 2023-12-16T21:42:59.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad740052b0 0x7fad8c13b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad8c0b12f0 0x7fad8c1dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8c0b2220 0x7fad8c135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c140f90 con 0x7fad8c0b2220 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c1df4e0 con 0x7fad740052b0 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad911b0640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0b2700 con 0x7fad8c0b12f0 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad8b59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8c0b2220 0x7fad8c135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad8b59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8c0b2220 0x7fad8c135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49386/0 (socket says 172.21.15.46:49386) 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.287+0000 7fad8b59e640 1 -- 172.21.15.46:0/3023173246 learned_addr learned my addr 172.21.15.46:0/3023173246 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8ad9d640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad740052b0 0x7fad8c13b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8a59c640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad8c0b12f0 0x7fad8c1dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8b59e640 1 -- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad8c0b12f0 msgr2=0x7fad8c1dfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8b59e640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad8c0b12f0 0x7fad8c1dfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8b59e640 1 -- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad740052b0 msgr2=0x7fad8c13b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8b59e640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad740052b0 0x7fad8c13b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8b59e640 1 -- 172.21.15.46:0/3023173246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0b1780 con 0x7fad8c0b2220 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8b59e640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8c0b2220 0x7fad8c135510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad8401cc40 tx=0x7fad74005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad737fe640 1 -- 172.21.15.46:0/3023173246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad840b1070 con 0x7fad8c0b2220 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad737fe640 1 -- 172.21.15.46:0/3023173246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad84003890 con 0x7fad8c0b2220 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad737fe640 1 -- 172.21.15.46:0/3023173246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad84003c00 con 0x7fad8c0b2220 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c1df6f0 con 0x7fad8c0b2220 2023-12-16T21:42:59.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c1e0150 con 0x7fad8c0b2220 2023-12-16T21:42:59.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c1df930 con 0x7fad8c0b2220 2023-12-16T21:42:59.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad737fe640 1 -- 172.21.15.46:0/3023173246 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fad840b6460 con 0x7fad8c0b2220 2023-12-16T21:42:59.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad737fe640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fad78066470 0x7fad78068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad737fe640 1 -- 172.21.15.46:0/3023173246 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad8411ca60 con 0x7fad8c0b2220 2023-12-16T21:42:59.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8ad9d640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fad78066470 0x7fad78068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.300 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.291+0000 7fad8ad9d640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fad78066470 0x7fad78068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0056c0 tx=0x7fad7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.295+0000 7fad737fe640 1 -- 172.21.15.46:0/3023173246 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fad8c1df930 con 0x7fad8c0b2220 2023-12-16T21:42:59.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.451+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fad8c1df930 con 0x7fad8c0b2220 2023-12-16T21:42:59.461 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad737fe640 1 -- 172.21.15.46:0/3023173246 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fad8c1df930 con 0x7fad8c0b2220 2023-12-16T21:42:59.461 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:42:59.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fad78066470 msgr2=0x7fad78068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fad78066470 0x7fad78068930 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0056c0 tx=0x7fad7c009040 comp rx=0 tx=0).stop 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8c0b2220 msgr2=0x7fad8c135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8c0b2220 0x7fad8c135510 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad8401cc40 tx=0x7fad74005800 comp rx=0 tx=0).stop 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 shutdown_connections 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fad78066470 0x7fad78068930 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad8c0b2220 0x7fad8c135510 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fad8c0b12f0 0x7fad8c1dfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 --2- 172.21.15.46:0/3023173246 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fad740052b0 0x7fad8c13b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 >> 172.21.15.46:0/3023173246 conn(0x7fad8c06a980 msgr2=0x7fad8c0b9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 shutdown_connections 2023-12-16T21:42:59.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.455+0000 7fad911b0640 1 -- 172.21.15.46:0/3023173246 wait complete. 2023-12-16T21:42:59.480 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:42:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f604650c640 1 Processor -- start 2023-12-16T21:42:59.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f604650c640 1 -- start start 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f604650c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60400aded0 0x7f6040143160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f604650c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f60401436a0 0x7f60400b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f604650c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60400ace20 con 0x7f60400b3390 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f604650c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6040143b50 con 0x7f60400aded0 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f604650c640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60400aeee0 con 0x7f60401436a0 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f6044aa9640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f60400b3390 0x7f60401460a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55128/0 (socket says 172.21.15.46:55128) 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f60400aded0 0x7f6040143160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f6044aa9640 1 -- 172.21.15.46:0/3106979167 learned_addr learned my addr 172.21.15.46:0/3106979167 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603effd640 1 -- 172.21.15.46:0/3106979167 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083776291 0 0) 0x7f60400ace20 con 0x7f60400b3390 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603f7fe640 1 --2- 172.21.15.46:0/3106979167 >> v2:172.21.15.161:3300/0 conn(0x7f60401436a0 0x7f60400b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603effd640 1 -- 172.21.15.46:0/3106979167 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6028003530 con 0x7f60400b3390 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603ffff640 1 -- 172.21.15.46:0/3106979167 >> v2:172.21.15.161:3300/0 conn(0x7f60401436a0 msgr2=0x7f60400b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603ffff640 1 --2- 172.21.15.46:0/3106979167 >> v2:172.21.15.161:3300/0 conn(0x7f60401436a0 0x7f60400b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603ffff640 1 -- 172.21.15.46:0/3106979167 >> v1:172.21.15.46:6789/0 conn(0x7f60400b3390 legacy=0x7f60401460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603ffff640 1 -- 172.21.15.46:0/3106979167 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60400ad070 con 0x7f60400aded0 2023-12-16T21:42:59.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.603+0000 7f603f7fe640 1 --2- 172.21.15.46:0/3106979167 >> v2:172.21.15.161:3300/0 conn(0x7f60401436a0 0x7f60400b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:59.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f603ffff640 1 --2- 172.21.15.46:0/3106979167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f6040143160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f604006d110 tx=0x7f603001c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0d1bf5ef1bc845 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f603effd640 1 -- 172.21.15.46:0/3106979167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300a5720 con 0x7f60400aded0 2023-12-16T21:42:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f603effd640 1 -- 172.21.15.46:0/3106979167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60300a5880 con 0x7f60400aded0 2023-12-16T21:42:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f603effd640 1 -- 172.21.15.46:0/3106979167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300a5c10 con 0x7f60400aded0 2023-12-16T21:42:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- 172.21.15.46:0/3106979167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 msgr2=0x7f6040143160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 --2- 172.21.15.46:0/3106979167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f6040143160 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f604006d110 tx=0x7f603001c7f0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- 172.21.15.46:0/3106979167 shutdown_connections 2023-12-16T21:42:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 --2- 172.21.15.46:0/3106979167 >> v2:172.21.15.161:3300/0 conn(0x7f60401436a0 0x7f60400b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 --2- 172.21.15.46:0/3106979167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f6040143160 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- 172.21.15.46:0/3106979167 >> 172.21.15.46:0/3106979167 conn(0x7f60400b4900 msgr2=0x7f60400b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- 172.21.15.46:0/3106979167 shutdown_connections 2023-12-16T21:42:59.615 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- 172.21.15.46:0/3106979167 wait complete. 2023-12-16T21:42:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 Processor -- start 2023-12-16T21:42:59.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- start start 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f60400719c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60400b3390 0x7f6040140d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f60401436a0 0x7f6040146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6040145d50 con 0x7f60400aded0 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6040151740 con 0x7f60400b3390 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f604650c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6040151a20 con 0x7f60401436a0 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f603ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f60400719c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f603ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f60400719c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49412/0 (socket says 172.21.15.46:49412) 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.607+0000 7f603ffff640 1 -- 172.21.15.46:0/429950105 learned_addr learned my addr 172.21.15.46:0/429950105 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f6044aa9640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f60401436a0 0x7f6040146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603f7fe640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60400b3390 0x7f6040140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603ffff640 1 -- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f60401436a0 msgr2=0x7f6040146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603ffff640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f60401436a0 0x7f6040146510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603ffff640 1 -- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60400b3390 msgr2=0x7f6040140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603ffff640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60400b3390 0x7f6040140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603ffff640 1 -- 172.21.15.46:0/429950105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6030009a70 con 0x7f60400aded0 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603f7fe640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60400b3390 0x7f6040140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603ffff640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f60400719c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f60300b0040 tx=0x7f6030019590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603cff9640 1 -- 172.21.15.46:0/429950105 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300ad850 con 0x7f60400aded0 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603cff9640 1 -- 172.21.15.46:0/429950105 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60300ad9d0 con 0x7f60400aded0 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603cff9640 1 -- 172.21.15.46:0/429950105 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60300add80 con 0x7f60400aded0 2023-12-16T21:42:59.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6040152360 con 0x7f60400aded0 2023-12-16T21:42:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6040146a50 con 0x7f60400aded0 2023-12-16T21:42:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6040068960 con 0x7f60400aded0 2023-12-16T21:42:59.620 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603cff9640 1 -- 172.21.15.46:0/429950105 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f60300b4050 con 0x7f60400aded0 2023-12-16T21:42:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603cff9640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f602c0664f0 0x7f602c0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603cff9640 1 -- 172.21.15.46:0/429950105 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f603011f140 con 0x7f60400aded0 2023-12-16T21:42:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.611+0000 7f603f7fe640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f602c0664f0 0x7f602c0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.621 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.615+0000 7f603f7fe640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f602c0664f0 0x7f602c0689b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6040145910 tx=0x7f6034008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.623 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.615+0000 7f603cff9640 1 -- 172.21.15.46:0/429950105 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f60300ec880 con 0x7f60400aded0 2023-12-16T21:42:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.775+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6040151b60 con 0x7f60400aded0 2023-12-16T21:42:59.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.775+0000 7f603cff9640 1 -- 172.21.15.46:0/429950105 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6040151b60 con 0x7f60400aded0 2023-12-16T21:42:59.784 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f602c0664f0 msgr2=0x7f602c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f602c0664f0 0x7f602c0689b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6040145910 tx=0x7f6034008040 comp rx=0 tx=0).stop 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 msgr2=0x7f60400719c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f60400719c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f60300b0040 tx=0x7f6030019590 comp rx=0 tx=0).stop 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 shutdown_connections 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f602c0664f0 0x7f602c0689b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f60401436a0 0x7f6040146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f60400b3390 0x7f6040140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 --2- 172.21.15.46:0/429950105 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60400aded0 0x7f60400719c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 >> 172.21.15.46:0/429950105 conn(0x7f60400b4900 msgr2=0x7f6040066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 shutdown_connections 2023-12-16T21:42:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.779+0000 7f604650c640 1 -- 172.21.15.46:0/429950105 wait complete. 2023-12-16T21:42:59.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:42:59.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3921647640 1 Processor -- start 2023-12-16T21:42:59.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3921647640 1 -- start start 2023-12-16T21:42:59.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3921647640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f391c0b5ec0 0x7f391c141620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3921647640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f391c141b60 0x7f391c14ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3921647640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391c0b1ba0 con 0x7f391c0ba920 2023-12-16T21:42:59.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3921647640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c141f90 con 0x7f391c0b5ec0 2023-12-16T21:42:59.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3921647640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391c0b4b50 con 0x7f391c141b60 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391affd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f391c0ba920 0x7f391c147a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55140/0 (socket says 172.21.15.46:55140) 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391affd640 1 -- 172.21.15.46:0/3303406510 learned_addr learned my addr 172.21.15.46:0/3303406510 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391a7fc640 1 --2- 172.21.15.46:0/3303406510 >> v2:172.21.15.46:3300/0 conn(0x7f391c0b5ec0 0x7f391c141620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391b7fe640 1 --2- 172.21.15.46:0/3303406510 >> v2:172.21.15.161:3300/0 conn(0x7f391c141b60 0x7f391c14ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3919ffb640 1 -- 172.21.15.46:0/3303406510 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700014707 0 0) 0x7f391c0b1ba0 con 0x7f391c0ba920 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f3919ffb640 1 -- 172.21.15.46:0/3303406510 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3904003530 con 0x7f391c0ba920 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391a7fc640 1 -- 172.21.15.46:0/3303406510 >> v2:172.21.15.161:3300/0 conn(0x7f391c141b60 msgr2=0x7f391c14ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391a7fc640 1 --2- 172.21.15.46:0/3303406510 >> v2:172.21.15.161:3300/0 conn(0x7f391c141b60 0x7f391c14ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391a7fc640 1 -- 172.21.15.46:0/3303406510 >> v1:172.21.15.46:6789/0 conn(0x7f391c0ba920 legacy=0x7f391c147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391a7fc640 1 -- 172.21.15.46:0/3303406510 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391c0b6350 con 0x7f391c0b5ec0 2023-12-16T21:42:59.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.919+0000 7f391b7fe640 1 --2- 172.21.15.46:0/3303406510 >> v2:172.21.15.161:3300/0 conn(0x7f391c141b60 0x7f391c14ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:42:59.929 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 --2- 172.21.15.46:0/3303406510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0b5ec0 0x7f391c141620 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f391c0b59b0 tx=0x7f391001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302796442d56f059 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3919ffb640 1 -- 172.21.15.46:0/3303406510 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39100a5960 con 0x7f391c0b5ec0 2023-12-16T21:42:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3919ffb640 1 -- 172.21.15.46:0/3303406510 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39100a5ac0 con 0x7f391c0b5ec0 2023-12-16T21:42:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3919ffb640 1 -- 172.21.15.46:0/3303406510 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39100a5e50 con 0x7f391c0b5ec0 2023-12-16T21:42:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- 172.21.15.46:0/3303406510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0b5ec0 msgr2=0x7f391c141620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 --2- 172.21.15.46:0/3303406510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0b5ec0 0x7f391c141620 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f391c0b59b0 tx=0x7f391001c880 comp rx=0 tx=0).stop 2023-12-16T21:42:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- 172.21.15.46:0/3303406510 shutdown_connections 2023-12-16T21:42:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 --2- 172.21.15.46:0/3303406510 >> v2:172.21.15.161:3300/0 conn(0x7f391c141b60 0x7f391c14ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 --2- 172.21.15.46:0/3303406510 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0b5ec0 0x7f391c141620 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- 172.21.15.46:0/3303406510 >> 172.21.15.46:0/3303406510 conn(0x7f391c06ab80 msgr2=0x7f391c0b9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:42:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- 172.21.15.46:0/3303406510 shutdown_connections 2023-12-16T21:42:59.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- 172.21.15.46:0/3303406510 wait complete. 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 Processor -- start 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- start start 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c0b5ec0 0x7f391c0b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0ba920 0x7f391c1e0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f391c141b60 0x7f391c138240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1dfed0 con 0x7f391c0ba920 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391c138da0 con 0x7f391c0b5ec0 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f3921647640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391c1df910 con 0x7f391c141b60 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0ba920 0x7f391c1e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0ba920 0x7f391c1e0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49422/0 (socket says 172.21.15.46:49422) 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 -- 172.21.15.46:0/2186270877 learned_addr learned my addr 172.21.15.46:0/2186270877 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:42:59.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 -- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f391c141b60 msgr2=0x7f391c138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f391c141b60 0x7f391c138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 -- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c0b5ec0 msgr2=0x7f391c0b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c0b5ec0 0x7f391c0b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 -- 172.21.15.46:0/2186270877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3910009a70 con 0x7f391c0ba920 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.923+0000 7f391a7fc640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0ba920 0x7f391c1e0360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3910019490 tx=0x7f3910019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f39037fe640 1 -- 172.21.15.46:0/2186270877 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39100b7610 con 0x7f391c0ba920 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f39037fe640 1 -- 172.21.15.46:0/2186270877 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39100b7770 con 0x7f391c0ba920 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f39037fe640 1 -- 172.21.15.46:0/2186270877 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39100b7b00 con 0x7f391c0ba920 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391c0728a0 con 0x7f391c0ba920 2023-12-16T21:42:59.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391c1390f0 con 0x7f391c0ba920 2023-12-16T21:42:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391c068960 con 0x7f391c0ba920 2023-12-16T21:42:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f39037fe640 1 -- 172.21.15.46:0/2186270877 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f39100b4050 con 0x7f391c0ba920 2023-12-16T21:42:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f39037fe640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f39080664a0 0x7f3908068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:42:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f39037fe640 1 -- 172.21.15.46:0/2186270877 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f391011f8d0 con 0x7f391c0ba920 2023-12-16T21:42:59.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f391affd640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f39080664a0 0x7f3908068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:42:59.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.927+0000 7f391affd640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f39080664a0 0x7f3908068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3914007fd0 tx=0x7f3914007900 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:42:59.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:42:59.931+0000 7f39037fe640 1 -- 172.21.15.46:0/2186270877 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f39100ed0a0 con 0x7f391c0ba920 2023-12-16T21:43:00.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.087+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f391c13f350 con 0x7f391c0ba920 2023-12-16T21:43:00.096 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.087+0000 7f39037fe640 1 -- 172.21.15.46:0/2186270877 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39100b2020 con 0x7f391c0ba920 2023-12-16T21:43:00.096 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:43:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f39080664a0 msgr2=0x7f3908068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f39080664a0 0x7f3908068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3914007fd0 tx=0x7f3914007900 comp rx=0 tx=0).stop 2023-12-16T21:43:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0ba920 msgr2=0x7f391c1e0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.098 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0ba920 0x7f391c1e0360 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3910019490 tx=0x7f3910019620 comp rx=0 tx=0).stop 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 shutdown_connections 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f39080664a0 0x7f3908068960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f391c141b60 0x7f391c138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f391c0ba920 0x7f391c1e0360 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 --2- 172.21.15.46:0/2186270877 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f391c0b5ec0 0x7f391c0b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 >> 172.21.15.46:0/2186270877 conn(0x7f391c06ab80 msgr2=0x7f391c1481d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 shutdown_connections 2023-12-16T21:43:00.099 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.091+0000 7f3921647640 1 -- 172.21.15.46:0/2186270877 wait complete. 2023-12-16T21:43:00.137 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2023-12-16T21:43:00.138 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2023-12-16T21:43:00.138 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:00.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.247+0000 7f594860a640 1 Processor -- start 2023-12-16T21:43:00.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.247+0000 7f594860a640 1 -- start start 2023-12-16T21:43:00.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f594860a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59400b3fd0 0x7f594013e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f594860a640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59400b0340 0x7f594013e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f594860a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400af2e0 con 0x7f594006c920 2023-12-16T21:43:00.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f594860a640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400b1f50 con 0x7f59400b0340 2023-12-16T21:43:00.257 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f594860a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594013eda0 con 0x7f59400b3fd0 2023-12-16T21:43:00.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59463a6640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f594006c920 0x7f59400afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55144/0 (socket says 172.21.15.46:55144) 2023-12-16T21:43:00.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59463a6640 1 -- 172.21.15.46:0/1150501622 learned_addr learned my addr 172.21.15.46:0/1150501622 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:00.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5946ba7640 1 --2- 172.21.15.46:0/1150501622 >> v2:172.21.15.161:3300/0 conn(0x7f59400b0340 0x7f594013e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.258 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59400b3fd0 0x7f594013e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879639566 0 0) 0x7f59400af2e0 con 0x7f594006c920 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5928003530 con 0x7f594006c920 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 232815396 0 0) 0x7f5928003530 con 0x7f594006c920 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59400b3fd0 msgr2=0x7f594013e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59400b3fd0 0x7f594013e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 >> v2:172.21.15.161:3300/0 conn(0x7f59400b0340 msgr2=0x7f594013e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 --2- 172.21.15.46:0/1150501622 >> v2:172.21.15.161:3300/0 conn(0x7f59400b0340 0x7f594013e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594006cd00 con 0x7f594006c920 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f5938001a00 con 0x7f594006c920 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59400b3fd0 0x7f594013e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:00.260 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5946ba7640 1 --2- 172.21.15.46:0/1150501622 >> v2:172.21.15.161:3300/0 conn(0x7f59400b0340 0x7f594013e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 >> v1:172.21.15.46:6789/0 conn(0x7f594006c920 legacy=0x7f59400afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5928003980 0x7f5928003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59280042a0 0x7f59280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5928004e20 0x7f59280052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59400b1f50 con 0x7f59280042a0 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594013eda0 con 0x7f5928004e20 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592800bd70 con 0x7f5928003980 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59280042a0 0x7f59280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f59463a6640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5928003980 0x7f5928003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5946ba7640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5928004e20 0x7f59280052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 -- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5928003980 msgr2=0x7f5928003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.261 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f5928003980 0x7f5928003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 -- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5928004e20 msgr2=0x7f59280052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f5928004e20 0x7f59280052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.262 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.251+0000 7f5945ba5640 1 -- 172.21.15.46:0/1150501622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593c009420 con 0x7f59280042a0 2023-12-16T21:43:00.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59280042a0 0x7f59280047d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f593c009cb0 tx=0x7f593c01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d169fc0cc4a0ea0e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0a57a0 con 0x7f59280042a0 2023-12-16T21:43:00.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c0a5900 con 0x7f59280042a0 2023-12-16T21:43:00.264 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f59453a4640 1 -- 172.21.15.46:0/1150501622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0a5cf0 con 0x7f59280042a0 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- 172.21.15.46:0/1150501622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59280042a0 msgr2=0x7f59280047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 --2- 172.21.15.46:0/1150501622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59280042a0 0x7f59280047d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f593c009cb0 tx=0x7f593c01cbf0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5946ba7640 1 -- 172.21.15.46:0/1150501622 reap_dead start 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- 172.21.15.46:0/1150501622 shutdown_connections 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- 172.21.15.46:0/1150501622 >> 172.21.15.46:0/1150501622 conn(0x7f59400b48e0 msgr2=0x7f59400b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- 172.21.15.46:0/1150501622 shutdown_connections 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- 172.21.15.46:0/1150501622 wait complete. 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 Processor -- start 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- start start 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f594014a010 0x7f5940140a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5940140f60 0x7f594013b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f594013b600 0x7f59400acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5940141f40 con 0x7f5940140f60 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594013ffb0 con 0x7f594013b600 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594013bab0 con 0x7f594014a010 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5940140f60 0x7f594013b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5940140f60 0x7f594013b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49436/0 (socket says 172.21.15.46:49436) 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 -- 172.21.15.46:0/2049449177 learned_addr learned my addr 172.21.15.46:0/2049449177 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5946ba7640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f594013b600 0x7f59400acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f59463a6640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f594014a010 0x7f5940140a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 -- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f594014a010 msgr2=0x7f5940140a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f594014a010 0x7f5940140a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 -- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f594013b600 msgr2=0x7f59400acf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f594013b600 0x7f59400acf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 -- 172.21.15.46:0/2049449177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59401413e0 con 0x7f5940140f60 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5946ba7640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f594013b600 0x7f59400acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5945ba5640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5940140f60 0x7f594013b0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f593c01d490 tx=0x7f593c019990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5936ffd640 1 -- 172.21.15.46:0/2049449177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0b8450 con 0x7f5940140f60 2023-12-16T21:43:00.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5936ffd640 1 -- 172.21.15.46:0/2049449177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c0b85b0 con 0x7f5940140f60 2023-12-16T21:43:00.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f5936ffd640 1 -- 172.21.15.46:0/2049449177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c0b8960 con 0x7f5940140f60 2023-12-16T21:43:00.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594006ea50 con 0x7f5940140f60 2023-12-16T21:43:00.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.255+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59400aee80 con 0x7f5940140f60 2023-12-16T21:43:00.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.259+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59400679e0 con 0x7f5940140f60 2023-12-16T21:43:00.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.259+0000 7f5936ffd640 1 -- 172.21.15.46:0/2049449177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f593c0b4070 con 0x7f5940140f60 2023-12-16T21:43:00.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.259+0000 7f5936ffd640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f592c0664a0 0x7f592c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.259+0000 7f5936ffd640 1 -- 172.21.15.46:0/2049449177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f593c11d970 con 0x7f5940140f60 2023-12-16T21:43:00.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.259+0000 7f59463a6640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f592c0664a0 0x7f592c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.259+0000 7f59463a6640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f592c0664a0 0x7f592c068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f594009dcc0 tx=0x7f5938009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.263+0000 7f5936ffd640 1 -- 172.21.15.46:0/2049449177 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f593c0eafa0 con 0x7f5940140f60 2023-12-16T21:43:00.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.419+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59400688f0 con 0x7f5940140f60 2023-12-16T21:43:00.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.423+0000 7f5936ffd640 1 -- 172.21.15.46:0/2049449177 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f593c0b2020 con 0x7f5940140f60 2023-12-16T21:43:00.431 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:00.431 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:00.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f592c0664a0 msgr2=0x7f592c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f592c0664a0 0x7f592c068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f594009dcc0 tx=0x7f5938009040 comp rx=0 tx=0).stop 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5940140f60 msgr2=0x7f594013b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5940140f60 0x7f594013b0c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f593c01d490 tx=0x7f593c019990 comp rx=0 tx=0).stop 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 shutdown_connections 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f592c0664a0 0x7f592c068960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f594013b600 0x7f59400acf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5940140f60 0x7f594013b0c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 --2- 172.21.15.46:0/2049449177 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f594014a010 0x7f5940140a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 >> 172.21.15.46:0/2049449177 conn(0x7f59400b48e0 msgr2=0x7f594014e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 shutdown_connections 2023-12-16T21:43:00.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.427+0000 7f594860a640 1 -- 172.21.15.46:0/2049449177 wait complete. 2023-12-16T21:43:00.451 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:43:00.452 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:43:00.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.575+0000 7fc6658f7640 1 Processor -- start 2023-12-16T21:43:00.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.575+0000 7fc6658f7640 1 -- start start 2023-12-16T21:43:00.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.575+0000 7fc6658f7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 0x7fc6600aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.575+0000 7fc6658f7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6600b1730 con 0x7fc6601451c0 2023-12-16T21:43:00.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.575+0000 7fc6658f7640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66006c7f0 con 0x7fc6600af300 2023-12-16T21:43:00.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.575+0000 7fc6658f7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66006cd70 con 0x7fc6600b1eb0 2023-12-16T21:43:00.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65f7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc6601451c0 0x7fc66014ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55158/0 (socket says 172.21.15.46:55158) 2023-12-16T21:43:00.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65f7fe640 1 -- 172.21.15.46:0/1321633916 learned_addr learned my addr 172.21.15.46:0/1321633916 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:00.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 0x7fc6600aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972717561 0 0) 0x7fc6600b1730 con 0x7fc6601451c0 2023-12-16T21:43:00.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc648003530 con 0x7fc6601451c0 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2978428100 0 0) 0x7fc648003530 con 0x7fc6601451c0 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 msgr2=0x7fc6600aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 0x7fc6600aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 >> v1:172.21.15.161:6789/0 conn(0x7fc6600af300 legacy=0x7fc660144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6600b2290 con 0x7fc6601451c0 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fc650001680 con 0x7fc6601451c0 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 >> v1:172.21.15.46:6789/0 conn(0x7fc6601451c0 legacy=0x7fc66014ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6480039b0 0x7fc648003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 0x7fc6480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc648004d10 0x7fc648005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66006cd70 con 0x7fc6480039b0 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64800bb20 con 0x7fc648004d10 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64800bcb0 con 0x7fc6600b1eb0 2023-12-16T21:43:00.587 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6480039b0 0x7fc648003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc648004d10 0x7fc648005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 -- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 msgr2=0x7fc6480047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65e7fc640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 0x7fc6480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6600b1eb0 0x7fc6480047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 -- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc648004d10 msgr2=0x7fc648005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc648004d10 0x7fc648005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 -- 172.21.15.46:0/1321633916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648003530 con 0x7fc6480039b0 2023-12-16T21:43:00.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc648004d10 0x7fc648005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65effd640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6480039b0 0x7fc648003d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc65800e170 tx=0x7fc65801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06b5f9ff0fb4667 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580a5a40 con 0x7fc6480039b0 2023-12-16T21:43:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65800f040 con 0x7fc6480039b0 2023-12-16T21:43:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.579+0000 7fc65dffb640 1 -- 172.21.15.46:0/1321633916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580a5de0 con 0x7fc6480039b0 2023-12-16T21:43:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/1321633916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6480039b0 msgr2=0x7fc648003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 --2- 172.21.15.46:0/1321633916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6480039b0 0x7fc648003d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc65800e170 tx=0x7fc65801cc40 comp rx=0 tx=0).stop 2023-12-16T21:43:00.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 -- 172.21.15.46:0/1321633916 reap_dead start 2023-12-16T21:43:00.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/1321633916 shutdown_connections 2023-12-16T21:43:00.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/1321633916 >> 172.21.15.46:0/1321633916 conn(0x7fc6600b4900 msgr2=0x7fc6600b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:00.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/1321633916 shutdown_connections 2023-12-16T21:43:00.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/1321633916 wait complete. 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 Processor -- start 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- start start 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6480039b0 0x7fc66014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6601e1fd0 0x7fc660136360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6601368a0 0x7fc660137870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc660133ee0 con 0x7fc6601368a0 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc660134bc0 con 0x7fc6480039b0 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600ae1b0 con 0x7fc6601e1fd0 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6601368a0 0x7fc660137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6601368a0 0x7fc660137870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49448/0 (socket says 172.21.15.46:49448) 2023-12-16T21:43:00.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 -- 172.21.15.46:0/215165612 learned_addr learned my addr 172.21.15.46:0/215165612 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65effd640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6480039b0 0x7fc66014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65e7fc640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6601e1fd0 0x7fc660136360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 -- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6601e1fd0 msgr2=0x7fc660136360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6601e1fd0 0x7fc660136360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 -- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6480039b0 msgr2=0x7fc66014f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6480039b0 0x7fc66014f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 -- 172.21.15.46:0/215165612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660134e30 con 0x7fc6601368a0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc65f7fe640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6601368a0 0x7fc660137870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc6600b5920 tx=0x7fc65000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc647fff640 1 -- 172.21.15.46:0/215165612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000eed0 con 0x7fc6601368a0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc647fff640 1 -- 172.21.15.46:0/215165612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65000c610 con 0x7fc6601368a0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6600ae450 con 0x7fc6601368a0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc660141f30 con 0x7fc6601368a0 2023-12-16T21:43:00.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc647fff640 1 -- 172.21.15.46:0/215165612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000c9a0 con 0x7fc6601368a0 2023-12-16T21:43:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.583+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc640005260 con 0x7fc6601368a0 2023-12-16T21:43:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.587+0000 7fc647fff640 1 -- 172.21.15.46:0/215165612 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fc65000cb00 con 0x7fc6601368a0 2023-12-16T21:43:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.587+0000 7fc647fff640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc64c0664a0 0x7fc64c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.587+0000 7fc647fff640 1 -- 172.21.15.46:0/215165612 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6500839b0 con 0x7fc6601368a0 2023-12-16T21:43:00.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.587+0000 7fc65effd640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc64c0664a0 0x7fc64c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.587+0000 7fc65effd640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc64c0664a0 0x7fc64c068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6580af000 tx=0x7fc6580a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.598 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.591+0000 7fc647fff640 1 -- 172.21.15.46:0/215165612 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fc650050ec0 con 0x7fc6601368a0 2023-12-16T21:43:00.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.747+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc640005a90 con 0x7fc6601368a0 2023-12-16T21:43:00.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.747+0000 7fc647fff640 1 -- 172.21.15.46:0/215165612 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6500544e0 con 0x7fc6601368a0 2023-12-16T21:43:00.755 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc64c0664a0 msgr2=0x7fc64c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc64c0664a0 0x7fc64c068960 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc6580af000 tx=0x7fc6580a7040 comp rx=0 tx=0).stop 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6601368a0 msgr2=0x7fc660137870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6601368a0 0x7fc660137870 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc6600b5920 tx=0x7fc65000eb90 comp rx=0 tx=0).stop 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 shutdown_connections 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc64c0664a0 0x7fc64c068960 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc6601368a0 0x7fc660137870 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc6601e1fd0 0x7fc660136360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 --2- 172.21.15.46:0/215165612 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc6480039b0 0x7fc66014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 >> 172.21.15.46:0/215165612 conn(0x7fc6600b4900 msgr2=0x7fc660142f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 shutdown_connections 2023-12-16T21:43:00.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.751+0000 7fc6658f7640 1 -- 172.21.15.46:0/215165612 wait complete. 2023-12-16T21:43:00.774 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:43:00.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.891+0000 7f9e1da5b640 1 Processor -- start 2023-12-16T21:43:00.900 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.891+0000 7f9e1da5b640 1 -- start start 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e1da5b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 0x7f9e180b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e1da5b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e180b02e0 con 0x7f9e180ae750 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e1da5b640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e180adc40 con 0x7f9e180af300 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e1da5b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180aded0 con 0x7f9e1806c8b0 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9e180ae750 0x7f9e180ad530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55160/0 (socket says 172.21.15.46:55160) 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 -- 172.21.15.46:0/3028956865 learned_addr learned my addr 172.21.15.46:0/3028956865 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e16ffd640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 0x7f9e180b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1471945915 0 0) 0x7f9e180b02e0 con 0x7f9e180ae750 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e04005740 con 0x7f9e180ae750 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 636236866 0 0) 0x7f9e04005740 con 0x7f9e180ae750 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 msgr2=0x7f9e180b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 0x7f9e180b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 >> v1:172.21.15.161:6789/0 conn(0x7f9e180af300 legacy=0x7f9e180ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1806cc90 con 0x7f9e180ae750 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f9e10001680 con 0x7f9e180ae750 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 >> v1:172.21.15.46:6789/0 conn(0x7f9e180ae750 legacy=0x7f9e180ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e04005bc0 0x7f9e04005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 0x7f9e040069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e04006f20 0x7f9e040073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e180aded0 con 0x7f9e04006f20 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0400dd60 con 0x7f9e04005bc0 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0400df20 con 0x7f9e1806c8b0 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e04006f20 0x7f9e040073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e16ffd640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e04005bc0 0x7f9e04005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e167fc640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 0x7f9e040069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.904 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 -- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 msgr2=0x7f9e040069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e1806c8b0 0x7f9e040069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 -- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e04005bc0 msgr2=0x7f9e04005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:00.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e04005bc0 0x7f9e04005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.895+0000 7f9e177fe640 1 -- 172.21.15.46:0/3028956865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e04005740 con 0x7f9e04006f20 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e177fe640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e04006f20 0x7f9e040073a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c240 tx=0x7f9e1001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4347db453ae433 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10005ae0 con 0x7f9e04006f20 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e10005c60 con 0x7f9e04006f20 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e15ffb640 1 -- 172.21.15.46:0/3028956865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e100a5810 con 0x7f9e04006f20 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- 172.21.15.46:0/3028956865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e04006f20 msgr2=0x7f9e040073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 --2- 172.21.15.46:0/3028956865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e04006f20 0x7f9e040073a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000c240 tx=0x7f9e1001cc10 comp rx=0 tx=0).stop 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e177fe640 1 -- 172.21.15.46:0/3028956865 reap_dead start 2023-12-16T21:43:00.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- 172.21.15.46:0/3028956865 shutdown_connections 2023-12-16T21:43:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- 172.21.15.46:0/3028956865 >> 172.21.15.46:0/3028956865 conn(0x7f9e180b4900 msgr2=0x7f9e180b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- 172.21.15.46:0/3028956865 shutdown_connections 2023-12-16T21:43:00.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- 172.21.15.46:0/3028956865 wait complete. 2023-12-16T21:43:00.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 Processor -- start 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- start start 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1813d430 0x7f9e18144d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e18139fd0 0x7f9e1813a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e181ceda0 0x7f9e181cf220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18070950 con 0x7f9e1813d430 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18071170 con 0x7f9e181ceda0 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18070d20 con 0x7f9e18139fd0 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1813d430 0x7f9e18144d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1813d430 0x7f9e18144d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49466/0 (socket says 172.21.15.46:49466) 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 -- 172.21.15.46:0/1869946036 learned_addr learned my addr 172.21.15.46:0/1869946036 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e177fe640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e181ceda0 0x7f9e181cf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e167fc640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e18139fd0 0x7f9e1813a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 -- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e18139fd0 msgr2=0x7f9e1813a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e18139fd0 0x7f9e1813a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 -- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e181ceda0 msgr2=0x7f9e181cf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e181ceda0 0x7f9e181cf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 -- 172.21.15.46:0/1869946036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e1813a990 con 0x7f9e1813d430 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e16ffd640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1813d430 0x7f9e18144d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e000115e0 tx=0x7f9e00011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e0ffff640 1 -- 172.21.15.46:0/1869946036 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00015870 con 0x7f9e1813d430 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e0ffff640 1 -- 172.21.15.46:0/1869946036 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e000159d0 con 0x7f9e1813d430 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1813d810 con 0x7f9e1813d430 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e0ffff640 1 -- 172.21.15.46:0/1869946036 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00015d40 con 0x7f9e1813d430 2023-12-16T21:43:00.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.899+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1809db20 con 0x7f9e1813d430 2023-12-16T21:43:00.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.903+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e18070430 con 0x7f9e1813d430 2023-12-16T21:43:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.903+0000 7f9e0ffff640 1 -- 172.21.15.46:0/1869946036 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f9e00027420 con 0x7f9e1813d430 2023-12-16T21:43:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.903+0000 7f9e0ffff640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9df80664a0 0x7f9df8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.903+0000 7f9e0ffff640 1 -- 172.21.15.46:0/1869946036 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e00084470 con 0x7f9e1813d430 2023-12-16T21:43:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.903+0000 7f9e167fc640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9df80664a0 0x7f9df8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:00.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.903+0000 7f9e167fc640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9df80664a0 0x7f9df8068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000f90 tx=0x7f9e08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:00.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:00.907+0000 7f9e0ffff640 1 -- 172.21.15.46:0/1869946036 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f9e00051b10 con 0x7f9e1813d430 2023-12-16T21:43:01.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.059+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e18070ef0 con 0x7f9e1813d430 2023-12-16T21:43:01.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.059+0000 7f9e0ffff640 1 -- 172.21.15.46:0/1869946036 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e00055130 con 0x7f9e1813d430 2023-12-16T21:43:01.067 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:43:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9df80664a0 msgr2=0x7f9df8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9df80664a0 0x7f9df8068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9e08000f90 tx=0x7f9e08009040 comp rx=0 tx=0).stop 2023-12-16T21:43:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1813d430 msgr2=0x7f9e18144d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.069 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1813d430 0x7f9e18144d40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e000115e0 tx=0x7f9e00011aa0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 shutdown_connections 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9df80664a0 0x7f9df8068960 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9e181ceda0 0x7f9e181cf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9e18139fd0 0x7f9e1813a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 --2- 172.21.15.46:0/1869946036 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9e1813d430 0x7f9e18144d40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 >> 172.21.15.46:0/1869946036 conn(0x7f9e180b4900 msgr2=0x7f9e180b7240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 shutdown_connections 2023-12-16T21:43:01.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.063+0000 7f9e1da5b640 1 -- 172.21.15.46:0/1869946036 wait complete. 2023-12-16T21:43:01.085 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:43:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.195+0000 7f177ab47640 1 Processor -- start 2023-12-16T21:43:01.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.195+0000 7f177ab47640 1 -- start start 2023-12-16T21:43:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.195+0000 7f177ab47640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740af300 0x7f17740b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.195+0000 7f177ab47640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f17740ae750 0x7f17740ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.195+0000 7f177ab47640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17740b02e0 con 0x7f177406c8b0 2023-12-16T21:43:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.195+0000 7f177ab47640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17740aded0 con 0x7f17740ae750 2023-12-16T21:43:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.195+0000 7f177ab47640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177406cc90 con 0x7f17740af300 2023-12-16T21:43:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17788e3640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f177406c8b0 0x7f17740ace20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55172/0 (socket says 172.21.15.46:55172) 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17788e3640 1 -- 172.21.15.46:0/3676798931 learned_addr learned my addr 172.21.15.46:0/3676798931 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17790e4640 1 --2- 172.21.15.46:0/3676798931 >> v2:172.21.15.46:3300/0 conn(0x7f17740ae750 0x7f17740ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f1773fff640 1 --2- 172.21.15.46:0/3676798931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740af300 0x7f17740b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17737fe640 1 -- 172.21.15.46:0/3676798931 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116958620 0 0) 0x7f17740b02e0 con 0x7f177406c8b0 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17737fe640 1 -- 172.21.15.46:0/3676798931 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f175c005740 con 0x7f177406c8b0 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17790e4640 1 -- 172.21.15.46:0/3676798931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740af300 msgr2=0x7f17740b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17790e4640 1 --2- 172.21.15.46:0/3676798931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740af300 0x7f17740b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17790e4640 1 -- 172.21.15.46:0/3676798931 >> v1:172.21.15.46:6789/0 conn(0x7f177406c8b0 legacy=0x7f17740ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17790e4640 1 -- 172.21.15.46:0/3676798931 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17740b05c0 con 0x7f17740ae750 2023-12-16T21:43:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17790e4640 1 --2- 172.21.15.46:0/3676798931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17740ae750 0x7f17740ad530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f176c009d90 tx=0x7f176c01c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e808f771a985c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17737fe640 1 -- 172.21.15.46:0/3676798931 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c0a57c0 con 0x7f17740ae750 2023-12-16T21:43:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17737fe640 1 -- 172.21.15.46:0/3676798931 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176c0a5920 con 0x7f17740ae750 2023-12-16T21:43:01.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f17737fe640 1 -- 172.21.15.46:0/3676798931 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176c0a5cb0 con 0x7f17740ae750 2023-12-16T21:43:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 -- 172.21.15.46:0/3676798931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17740ae750 msgr2=0x7f17740ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 --2- 172.21.15.46:0/3676798931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17740ae750 0x7f17740ad530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f176c009d90 tx=0x7f176c01c800 comp rx=0 tx=0).stop 2023-12-16T21:43:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 -- 172.21.15.46:0/3676798931 shutdown_connections 2023-12-16T21:43:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 --2- 172.21.15.46:0/3676798931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f17740ae750 0x7f17740ad530 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 --2- 172.21.15.46:0/3676798931 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740af300 0x7f17740b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 -- 172.21.15.46:0/3676798931 >> 172.21.15.46:0/3676798931 conn(0x7f17740b4900 msgr2=0x7f17740b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:01.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 -- 172.21.15.46:0/3676798931 shutdown_connections 2023-12-16T21:43:01.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.199+0000 7f177ab47640 1 -- 172.21.15.46:0/3676798931 wait complete. 2023-12-16T21:43:01.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 Processor -- start 2023-12-16T21:43:01.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 -- start start 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177406c8b0 0x7f1774141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740ae750 0x7f1774142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17740af300 0x7f17741dd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17741da660 con 0x7f177406c8b0 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1774142560 con 0x7f17740af300 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17740702e0 con 0x7f17740ae750 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177406c8b0 0x7f1774141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177406c8b0 0x7f1774141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49494/0 (socket says 172.21.15.46:49494) 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 -- 172.21.15.46:0/1321500213 learned_addr learned my addr 172.21.15.46:0/1321500213 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 -- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740ae750 msgr2=0x7f1774142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740ae750 0x7f1774142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 -- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17740af300 msgr2=0x7f17741dd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.212 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17740af300 0x7f17741dd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 -- 172.21.15.46:0/1321500213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c009a70 con 0x7f177406c8b0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17788e3640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177406c8b0 0x7f1774141ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f176400c500 tx=0x7f176400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17717fa640 1 -- 172.21.15.46:0/1321500213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764005cf0 con 0x7f177406c8b0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17717fa640 1 -- 172.21.15.46:0/1321500213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764005e50 con 0x7f177406c8b0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1774149910 con 0x7f177406c8b0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17717fa640 1 -- 172.21.15.46:0/1321500213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17640072f0 con 0x7f177406c8b0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1774146cd0 con 0x7f177406c8b0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1774068960 con 0x7f177406c8b0 2023-12-16T21:43:01.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17717fa640 1 -- 172.21.15.46:0/1321500213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f1764007450 con 0x7f177406c8b0 2023-12-16T21:43:01.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17717fa640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f17600664a0 0x7f1760068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.203+0000 7f17717fa640 1 -- 172.21.15.46:0/1321500213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17640832f0 con 0x7f177406c8b0 2023-12-16T21:43:01.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.207+0000 7f1773fff640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f17600664a0 0x7f1760068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.207+0000 7f1773fff640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f17600664a0 0x7f1760068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17740b1ab0 tx=0x7f1768008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:01.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.211+0000 7f17717fa640 1 -- 172.21.15.46:0/1321500213 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f17640508b0 con 0x7f177406c8b0 2023-12-16T21:43:01.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17740b7400 con 0x7f177406c8b0 2023-12-16T21:43:01.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f17717fa640 1 -- 172.21.15.46:0/1321500213 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1764053ed0 con 0x7f177406c8b0 2023-12-16T21:43:01.374 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:43:01.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f17600664a0 msgr2=0x7f1760068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f17600664a0 0x7f1760068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17740b1ab0 tx=0x7f1768008040 comp rx=0 tx=0).stop 2023-12-16T21:43:01.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177406c8b0 msgr2=0x7f1774141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177406c8b0 0x7f1774141ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f176400c500 tx=0x7f176400c9c0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.376 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 shutdown_connections 2023-12-16T21:43:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f17600664a0 0x7f1760068960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f17740af300 0x7f17741dd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f17740ae750 0x7f1774142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 --2- 172.21.15.46:0/1321500213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f177406c8b0 0x7f1774141ae0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 >> 172.21.15.46:0/1321500213 conn(0x7f17740b4900 msgr2=0x7f1774145440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 shutdown_connections 2023-12-16T21:43:01.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.367+0000 7f177ab47640 1 -- 172.21.15.46:0/1321500213 wait complete. 2023-12-16T21:43:01.393 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2023-12-16T21:43:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc3c3f640 1 Processor -- start 2023-12-16T21:43:01.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc3c3f640 1 -- start start 2023-12-16T21:43:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc3c3f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdbbc06d190 0x7fdbbc0b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc3c3f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 0x7fdbbc140910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc3c3f640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0b73c0 con 0x7fdbbc0b6fe0 2023-12-16T21:43:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc3c3f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b5920 con 0x7fdbbc06d190 2023-12-16T21:43:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc3c3f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b4cb0 con 0x7fdbbc1404c0 2023-12-16T21:43:01.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc19db640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdbbc0b6fe0 0x7fdbbc06ca80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55188/0 (socket says 172.21.15.46:55188) 2023-12-16T21:43:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc19db640 1 -- 172.21.15.46:0/2966807847 learned_addr learned my addr 172.21.15.46:0/2966807847 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1144297583 0 0) 0x7fdbbc0b73c0 con 0x7fdbbc0b6fe0 2023-12-16T21:43:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdba8003530 con 0x7fdbbc0b6fe0 2023-12-16T21:43:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc11da640 1 --2- 172.21.15.46:0/2966807847 >> v2:172.21.15.161:3300/0 conn(0x7fdbbc06d190 0x7fdbbc0b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc21dc640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 0x7fdbbc140910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 61160287 0 0) 0x7fdba8003530 con 0x7fdbbc0b6fe0 2023-12-16T21:43:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 msgr2=0x7fdbbc140910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 0x7fdbbc140910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 >> v2:172.21.15.161:3300/0 conn(0x7fdbbc06d190 msgr2=0x7fdbbc0b6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 --2- 172.21.15.46:0/2966807847 >> v2:172.21.15.161:3300/0 conn(0x7fdbbc06d190 0x7fdbbc0b6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc0b4f90 con 0x7fdbbc0b6fe0 2023-12-16T21:43:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fdbb0001a20 con 0x7fdbbc0b6fe0 2023-12-16T21:43:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 >> v1:172.21.15.46:6789/0 conn(0x7fdbbc0b6fe0 legacy=0x7fdbbc06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 0x7fdba8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8003fb0 0x7fdba80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8004d10 0x7fdba8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b5920 con 0x7fdba8003fb0 2023-12-16T21:43:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b4cb0 con 0x7fdba8004d10 2023-12-16T21:43:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba800bb50 con 0x7fdbbc1404c0 2023-12-16T21:43:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc11da640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8003fb0 0x7fdba80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc21dc640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8004d10 0x7fdba8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc19db640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 0x7fdba8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc11da640 1 -- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 msgr2=0x7fdba8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc11da640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 0x7fdba8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc11da640 1 -- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8004d10 msgr2=0x7fdba8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc11da640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8004d10 0x7fdba8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc11da640 1 -- 172.21.15.46:0/2966807847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8003530 con 0x7fdba8003fb0 2023-12-16T21:43:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.511+0000 7fdbc19db640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1404c0 0x7fdba8003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:01.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc11da640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8003fb0 0x7fdba80047d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800ba30 tx=0x7fdbb801c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2586a262bfc799 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80ab670 con 0x7fdba8003fb0 2023-12-16T21:43:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80ab7d0 con 0x7fdba8003fb0 2023-12-16T21:43:01.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc09d9640 1 -- 172.21.15.46:0/2966807847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80abb80 con 0x7fdba8003fb0 2023-12-16T21:43:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/2966807847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8003fb0 msgr2=0x7fdba80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc3c3f640 1 --2- 172.21.15.46:0/2966807847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdba8003fb0 0x7fdba80047d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800ba30 tx=0x7fdbb801c640 comp rx=0 tx=0).stop 2023-12-16T21:43:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc21dc640 1 -- 172.21.15.46:0/2966807847 reap_dead start 2023-12-16T21:43:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/2966807847 shutdown_connections 2023-12-16T21:43:01.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/2966807847 >> 172.21.15.46:0/2966807847 conn(0x7fdbbc06a980 msgr2=0x7fdbbc06ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/2966807847 shutdown_connections 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.515+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/2966807847 wait complete. 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 Processor -- start 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 -- start start 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8003fb0 0x7fdbbc0ba630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbbc0b1950 0x7fdbbc1de6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1dec10 0x7fdbbc138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc1e26c0 con 0x7fdbbc0b1950 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b2010 con 0x7fdba8003fb0 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc1df670 con 0x7fdbbc1dec10 2023-12-16T21:43:01.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbbc0b1950 0x7fdbbc1de6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbbc0b1950 0x7fdbbc1de6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49508/0 (socket says 172.21.15.46:49508) 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 -- 172.21.15.46:0/821052141 learned_addr learned my addr 172.21.15.46:0/821052141 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc19db640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8003fb0 0x7fdbbc0ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc21dc640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1dec10 0x7fdbbc138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 -- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1dec10 msgr2=0x7fdbbc138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1dec10 0x7fdbbc138de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 -- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8003fb0 msgr2=0x7fdbbc0ba630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8003fb0 0x7fdbbc0ba630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 -- 172.21.15.46:0/821052141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc1e2ae0 con 0x7fdbbc0b1950 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc19db640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8003fb0 0x7fdbbc0ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc11da640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbbc0b1950 0x7fdbbc1de6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80ad040 tx=0x7fdba8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbae7fc640 1 -- 172.21.15.46:0/821052141 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8004220 con 0x7fdbbc0b1950 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbae7fc640 1 -- 172.21.15.46:0/821052141 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb80b8070 con 0x7fdbbc0b1950 2023-12-16T21:43:01.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc1e2000 con 0x7fdbbc0b1950 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbae7fc640 1 -- 172.21.15.46:0/821052141 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80ab940 con 0x7fdbbc0b1950 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc1e1c40 con 0x7fdbbc0b1950 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc0679e0 con 0x7fdbbc0b1950 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.519+0000 7fdbae7fc640 1 -- 172.21.15.46:0/821052141 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fdbb80abaa0 con 0x7fdbbc0b1950 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.523+0000 7fdbae7fc640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdba0066470 0x7fdba0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.523+0000 7fdbae7fc640 1 -- 172.21.15.46:0/821052141 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdbb811e730 con 0x7fdbbc0b1950 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.523+0000 7fdbc19db640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdba0066470 0x7fdba0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.523+0000 7fdbc19db640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdba0066470 0x7fdba0068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0007c50 tx=0x7fdbb0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:01.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.527+0000 7fdbae7fc640 1 -- 172.21.15.46:0/821052141 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fdbb80ebc40 con 0x7fdbbc0b1950 2023-12-16T21:43:01.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:01.743+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) v1 -- 0x7fdbbc067de0 con 0x7fdbbc0b1950 2023-12-16T21:43:02.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.523+0000 7fdbae7fc640 1 -- 172.21.15.46:0/821052141 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v45) v1 ==== 137+0+0 (secure 0 0 0) 0x7fdbb80ef260 con 0x7fdbbc0b1950 2023-12-16T21:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdba0066470 msgr2=0x7fdba0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdba0066470 0x7fdba0068930 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0007c50 tx=0x7fdbb0009040 comp rx=0 tx=0).stop 2023-12-16T21:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbbc0b1950 msgr2=0x7fdbbc1de6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbbc0b1950 0x7fdbbc1de6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdbb80ad040 tx=0x7fdba8005800 comp rx=0 tx=0).stop 2023-12-16T21:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 shutdown_connections 2023-12-16T21:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdba0066470 0x7fdba0068930 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdbbc1dec10 0x7fdbbc138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbbc0b1950 0x7fdbbc1de6d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 --2- 172.21.15.46:0/821052141 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdba8003fb0 0x7fdbbc0ba630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 >> 172.21.15.46:0/821052141 conn(0x7fdbbc06a980 msgr2=0x7fdbbc146800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 shutdown_connections 2023-12-16T21:43:02.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.527+0000 7fdbc3c3f640 1 -- 172.21.15.46:0/821052141 wait complete. 2023-12-16T21:43:02.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-16T21:43:02.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a274a5640 1 Processor -- start 2023-12-16T21:43:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a274a5640 1 -- start start 2023-12-16T21:43:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a274a5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4a201451c0 0x7f4a2014fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a274a5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a200b1730 con 0x7f4a200b1eb0 2023-12-16T21:43:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a274a5640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a2006c7f0 con 0x7f4a200af300 2023-12-16T21:43:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a274a5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2006cd70 con 0x7f4a201451c0 2023-12-16T21:43:02.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a25241640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4a200b1eb0 0x7f4a200aff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55194/0 (socket says 172.21.15.46:55194) 2023-12-16T21:43:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a25241640 1 -- 172.21.15.46:0/3126551354 learned_addr learned my addr 172.21.15.46:0/3126551354 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a25a42640 1 --2- 172.21.15.46:0/3126551354 >> v2:172.21.15.46:3300/0 conn(0x7f4a201451c0 0x7f4a2014fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a17fff640 1 -- 172.21.15.46:0/3126551354 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310127799 0 0) 0x7f4a200b1730 con 0x7f4a200b1eb0 2023-12-16T21:43:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a17fff640 1 -- 172.21.15.46:0/3126551354 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a0c003530 con 0x7f4a200b1eb0 2023-12-16T21:43:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a25a42640 1 -- 172.21.15.46:0/3126551354 >> v1:172.21.15.161:6789/0 conn(0x7f4a200af300 legacy=0x7f4a20144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a25a42640 1 -- 172.21.15.46:0/3126551354 >> v1:172.21.15.46:6789/0 conn(0x7f4a200b1eb0 legacy=0x7f4a200aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.693 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.683+0000 7f4a25a42640 1 -- 172.21.15.46:0/3126551354 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a200b2290 con 0x7f4a201451c0 2023-12-16T21:43:02.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25a42640 1 --2- 172.21.15.46:0/3126551354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a201451c0 0x7f4a2014fd20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009d90 tx=0x7f4a1801c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138db39ba9ded328 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a17fff640 1 -- 172.21.15.46:0/3126551354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180a5780 con 0x7f4a201451c0 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a17fff640 1 -- 172.21.15.46:0/3126551354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a180a58e0 con 0x7f4a201451c0 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a17fff640 1 -- 172.21.15.46:0/3126551354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a180a5c70 con 0x7f4a201451c0 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- 172.21.15.46:0/3126551354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a201451c0 msgr2=0x7f4a2014fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3126551354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a201451c0 0x7f4a2014fd20 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009d90 tx=0x7f4a1801c830 comp rx=0 tx=0).stop 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- 172.21.15.46:0/3126551354 shutdown_connections 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3126551354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a201451c0 0x7f4a2014fd20 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- 172.21.15.46:0/3126551354 >> 172.21.15.46:0/3126551354 conn(0x7f4a200b4900 msgr2=0x7f4a200b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- 172.21.15.46:0/3126551354 shutdown_connections 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- 172.21.15.46:0/3126551354 wait complete. 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 Processor -- start 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- start start 2023-12-16T21:43:02.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200af300 0x7f4a20137580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a200b1eb0 0x7f4a2006f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a201451c0 0x7f4a200ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2006fa10 con 0x7f4a200af300 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2006f020 con 0x7f4a200b1eb0 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a274a5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a20071230 con 0x7f4a201451c0 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200af300 0x7f4a20137580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200af300 0x7f4a20137580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49530/0 (socket says 172.21.15.46:49530) 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 -- 172.21.15.46:0/3234858939 learned_addr learned my addr 172.21.15.46:0/3234858939 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a24a40640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a200b1eb0 0x7f4a2006f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 -- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a201451c0 msgr2=0x7f4a200ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a201451c0 0x7f4a200ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 -- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a200b1eb0 msgr2=0x7f4a2006f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a200b1eb0 0x7f4a2006f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 -- 172.21.15.46:0/3234858939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a18009a70 con 0x7f4a200af300 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.687+0000 7f4a25241640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200af300 0x7f4a20137580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c500 tx=0x7f4a1c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a15ffb640 1 -- 172.21.15.46:0/3234858939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c005cf0 con 0x7f4a200af300 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a15ffb640 1 -- 172.21.15.46:0/3234858939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c005e50 con 0x7f4a200af300 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a15ffb640 1 -- 172.21.15.46:0/3234858939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0072f0 con 0x7f4a200af300 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2006fc50 con 0x7f4a200af300 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a2006e860 con 0x7f4a200af300 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a20068960 con 0x7f4a200af300 2023-12-16T21:43:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a15ffb640 1 -- 172.21.15.46:0/3234858939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f4a1c007450 con 0x7f4a200af300 2023-12-16T21:43:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a15ffb640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4a040664a0 0x7f4a04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a15ffb640 1 -- 172.21.15.46:0/3234858939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a1c0833c0 con 0x7f4a200af300 2023-12-16T21:43:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a24a40640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4a040664a0 0x7f4a04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:02.702 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.691+0000 7f4a24a40640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4a040664a0 0x7f4a04068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4a2009de80 tx=0x7f4a10009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:02.703 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.695+0000 7f4a15ffb640 1 -- 172.21.15.46:0/3234858939 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f4a1c050a20 con 0x7f4a200af300 2023-12-16T21:43:02.861 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.855+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4a200b1130 con 0x7f4a200af300 2023-12-16T21:43:02.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a15ffb640 1 -- 172.21.15.46:0/3234858939 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4a1c054040 con 0x7f4a200af300 2023-12-16T21:43:02.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4a040664a0 msgr2=0x7f4a04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4a040664a0 0x7f4a04068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4a2009de80 tx=0x7f4a10009040 comp rx=0 tx=0).stop 2023-12-16T21:43:02.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200af300 msgr2=0x7f4a20137580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200af300 0x7f4a20137580 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c500 tx=0x7f4a1c00c9c0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 shutdown_connections 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4a040664a0 0x7f4a04068960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4a201451c0 0x7f4a200ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4a200b1eb0 0x7f4a2006f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 --2- 172.21.15.46:0/3234858939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a200af300 0x7f4a20137580 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 >> 172.21.15.46:0/3234858939 conn(0x7f4a200b4900 msgr2=0x7f4a200b73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 shutdown_connections 2023-12-16T21:43:02.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:02.863+0000 7f4a274a5640 1 -- 172.21.15.46:0/3234858939 wait complete. 2023-12-16T21:43:02.888 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-16T21:43:02.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-16T21:43:02.950 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-16T21:43:03.003 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-16T21:43:03.003 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:43:03.003 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-16T21:43:03.008 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:43:03.009 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-16T21:43:03.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.223+0000 7f1425096640 1 Processor -- start 2023-12-16T21:43:03.232 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.223+0000 7f1425096640 1 -- start start 2023-12-16T21:43:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1425096640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1420147270 0x7f1420071f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1425096640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f142006ff60 0x7f14200b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1425096640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142006ac90 con 0x7f1420146690 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1425096640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14200724b0 con 0x7f1420147270 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1425096640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14200711b0 con 0x7f142006ff60 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f141ed9d640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1420147270 0x7f1420071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1417fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f142006ff60 0x7f14200b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f141f59e640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f1420146690 0x7f142006a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:49078/0 (socket says 172.21.15.46:49078) 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f141f59e640 1 -- 172.21.15.46:0/581045145 learned_addr learned my addr 172.21.15.46:0/581045145 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1417fff640 1 -- 172.21.15.46:0/581045145 >> v2:172.21.15.161:3300/0 conn(0x7f1420147270 msgr2=0x7f1420071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1417fff640 1 --2- 172.21.15.46:0/581045145 >> v2:172.21.15.161:3300/0 conn(0x7f1420147270 0x7f1420071f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f141e59c640 1 -- 172.21.15.46:0/581045145 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3644223128 0 0) 0x7f142006ac90 con 0x7f1420146690 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1417fff640 1 -- 172.21.15.46:0/581045145 >> v1:172.21.15.161:6789/0 conn(0x7f1420146690 legacy=0x7f142006a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f1417fff640 1 -- 172.21.15.46:0/581045145 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420147650 con 0x7f142006ff60 2023-12-16T21:43:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.227+0000 7f141ed9d640 1 --2- 172.21.15.46:0/581045145 >> v2:172.21.15.161:3300/0 conn(0x7f1420147270 0x7f1420071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:03.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1417fff640 1 --2- 172.21.15.46:0/581045145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f142006ff60 0x7f14200b4e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14080097c0 tx=0x7f140801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451d35d9ed016c82 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f141e59c640 1 -- 172.21.15.46:0/581045145 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14080ab600 con 0x7f142006ff60 2023-12-16T21:43:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f141e59c640 1 -- 172.21.15.46:0/581045145 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14080a7050 con 0x7f142006ff60 2023-12-16T21:43:03.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f141e59c640 1 -- 172.21.15.46:0/581045145 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14080ab980 con 0x7f142006ff60 2023-12-16T21:43:03.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- 172.21.15.46:0/581045145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f142006ff60 msgr2=0x7f14200b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 --2- 172.21.15.46:0/581045145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f142006ff60 0x7f14200b4e50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f14080097c0 tx=0x7f140801c3f0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- 172.21.15.46:0/581045145 shutdown_connections 2023-12-16T21:43:03.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 --2- 172.21.15.46:0/581045145 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f142006ff60 0x7f14200b4e50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 --2- 172.21.15.46:0/581045145 >> v2:172.21.15.161:3300/0 conn(0x7f1420147270 0x7f1420071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- 172.21.15.46:0/581045145 >> 172.21.15.46:0/581045145 conn(0x7f14200b5850 msgr2=0x7f14200b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:03.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- 172.21.15.46:0/581045145 shutdown_connections 2023-12-16T21:43:03.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- 172.21.15.46:0/581045145 wait complete. 2023-12-16T21:43:03.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 Processor -- start 2023-12-16T21:43:03.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- start start 2023-12-16T21:43:03.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1420146690 0x7f142014a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420147270 0x7f14200ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14200acea0 0x7f1420136de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14200af540 con 0x7f14200acea0 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1420148c70 con 0x7f1420147270 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f1425096640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14200ad350 con 0x7f1420146690 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f141f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14200acea0 0x7f1420136de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f141f59e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14200acea0 0x7f1420136de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49536/0 (socket says 172.21.15.46:49536) 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f141f59e640 1 -- 172.21.15.46:0/996331208 learned_addr learned my addr 172.21.15.46:0/996331208 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.231+0000 7f141ed9d640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1420146690 0x7f142014a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f1417fff640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420147270 0x7f14200ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f141f59e640 1 -- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1420146690 msgr2=0x7f142014a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f141f59e640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1420146690 0x7f142014a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f141f59e640 1 -- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420147270 msgr2=0x7f14200ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f141f59e640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420147270 0x7f14200ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f141f59e640 1 -- 172.21.15.46:0/996331208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1410009420 con 0x7f14200acea0 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f141f59e640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14200acea0 0x7f1420136de0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1418001570 tx=0x7f141800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f14177fe640 1 -- 172.21.15.46:0/996331208 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141800ce90 con 0x7f14200acea0 2023-12-16T21:43:03.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f14177fe640 1 -- 172.21.15.46:0/996331208 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141800a5e0 con 0x7f14200acea0 2023-12-16T21:43:03.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1408009420 con 0x7f14200acea0 2023-12-16T21:43:03.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f14177fe640 1 -- 172.21.15.46:0/996331208 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141800a9b0 con 0x7f14200acea0 2023-12-16T21:43:03.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1420149cb0 con 0x7f14200acea0 2023-12-16T21:43:03.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1420149000 con 0x7f14200acea0 2023-12-16T21:43:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f14177fe640 1 -- 172.21.15.46:0/996331208 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f141800ab10 con 0x7f14200acea0 2023-12-16T21:43:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f14177fe640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f14000664a0 0x7f1400068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f14177fe640 1 -- 172.21.15.46:0/996331208 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14180834e0 con 0x7f14200acea0 2023-12-16T21:43:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.235+0000 7f141ed9d640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f14000664a0 0x7f1400068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.239+0000 7f141ed9d640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f14000664a0 0x7f1400068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1410009c50 tx=0x7f1410009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.239+0000 7f14177fe640 1 -- 172.21.15.46:0/996331208 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f1418050970 con 0x7f14200acea0 2023-12-16T21:43:03.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.395+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1420149000 con 0x7f14200acea0 2023-12-16T21:43:03.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.399+0000 7f14177fe640 1 -- 172.21.15.46:0/996331208 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f1420149000 con 0x7f14200acea0 2023-12-16T21:43:03.410 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:03.410 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f14000664a0 msgr2=0x7f1400068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f14000664a0 0x7f1400068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1410009c50 tx=0x7f1410009190 comp rx=0 tx=0).stop 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14200acea0 msgr2=0x7f1420136de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14200acea0 0x7f1420136de0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1418001570 tx=0x7f141800cca0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 shutdown_connections 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f14000664a0 0x7f1400068960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14200acea0 0x7f1420136de0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1420147270 0x7f14200ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 --2- 172.21.15.46:0/996331208 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1420146690 0x7f142014a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 >> 172.21.15.46:0/996331208 conn(0x7f14200b5850 msgr2=0x7f1420145670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 shutdown_connections 2023-12-16T21:43:03.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.403+0000 7f1425096640 1 -- 172.21.15.46:0/996331208 wait complete. 2023-12-16T21:43:03.427 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:43:03.427 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:43:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe169c55640 1 Processor -- start 2023-12-16T21:43:03.557 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe169c55640 1 -- start start 2023-12-16T21:43:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe169c55640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe16406fc80 0x7fe164140000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe169c55640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe164140540 0x7fe164150f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe169c55640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1640700f0 con 0x7fe1640b5da0 2023-12-16T21:43:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe169c55640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe164070960 con 0x7fe16406fc80 2023-12-16T21:43:03.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe169c55640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe164071500 con 0x7fe164140540 2023-12-16T21:43:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe1640b5da0 0x7fe164149bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55210/0 (socket says 172.21.15.46:55210) 2023-12-16T21:43:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 -- 172.21.15.46:0/2242219444 learned_addr learned my addr 172.21.15.46:0/2242219444 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe15bfff640 1 --2- 172.21.15.46:0/2242219444 >> v2:172.21.15.161:3300/0 conn(0x7fe16406fc80 0x7fe164140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe163fff640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe164140540 0x7fe164150f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250719578 0 0) 0x7fe1640700f0 con 0x7fe1640b5da0 2023-12-16T21:43:03.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe150003530 con 0x7fe1640b5da0 2023-12-16T21:43:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3324928636 0 0) 0x7fe150003530 con 0x7fe1640b5da0 2023-12-16T21:43:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe164140540 msgr2=0x7fe164150f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe164140540 0x7fe164150f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 >> v2:172.21.15.161:3300/0 conn(0x7fe16406fc80 msgr2=0x7fe164140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:03.560 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 --2- 172.21.15.46:0/2242219444 >> v2:172.21.15.161:3300/0 conn(0x7fe16406fc80 0x7fe164140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe164071070 con 0x7fe1640b5da0 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fe154001680 con 0x7fe1640b5da0 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 >> v1:172.21.15.46:6789/0 conn(0x7fe1640b5da0 legacy=0x7fe164149bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe150003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1500047d0 0x7fe150004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe164140540 0x7fe150005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164071500 con 0x7fe150003980 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000b990 con 0x7fe1500047d0 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15000bb20 con 0x7fe164140540 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe150003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 -- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe164140540 msgr2=0x7fe150005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe164140540 0x7fe150005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.561 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 -- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1500047d0 msgr2=0x7fe150004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1500047d0 0x7fe150004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.562 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.551+0000 7fe1637fe640 1 -- 172.21.15.46:0/2242219444 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150003530 con 0x7fe150003980 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe1637fe640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe150003da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe15400c190 tx=0x7fe15401cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e574e0a122c5fb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1540058e0 con 0x7fe150003980 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe154005a40 con 0x7fe150003980 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe162ffd640 1 -- 172.21.15.46:0/2242219444 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154005df0 con 0x7fe150003980 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- 172.21.15.46:0/2242219444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 msgr2=0x7fe150003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 --2- 172.21.15.46:0/2242219444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe150003da0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe15400c190 tx=0x7fe15401cb80 comp rx=0 tx=0).stop 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe163fff640 1 -- 172.21.15.46:0/2242219444 reap_dead start 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- 172.21.15.46:0/2242219444 shutdown_connections 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- 172.21.15.46:0/2242219444 >> 172.21.15.46:0/2242219444 conn(0x7fe16406b860 msgr2=0x7fe16406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- 172.21.15.46:0/2242219444 shutdown_connections 2023-12-16T21:43:03.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- 172.21.15.46:0/2242219444 wait complete. 2023-12-16T21:43:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 Processor -- start 2023-12-16T21:43:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- start start 2023-12-16T21:43:03.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe1641e3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1640b00f0 0x7fe164148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1640b2850 0x7fe164145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640b0f10 con 0x7fe150003980 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe164135e40 con 0x7fe1640b2850 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe169c55640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1640b1be0 con 0x7fe1640b00f0 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe1641e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe1637fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe1641e3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49560/0 (socket says 172.21.15.46:49560) 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe1637fe640 1 -- 172.21.15.46:0/544233253 learned_addr learned my addr 172.21.15.46:0/544233253 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe163fff640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1640b2850 0x7fe164145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.555+0000 7fe15bfff640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1640b00f0 0x7fe164148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe1637fe640 1 -- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1640b00f0 msgr2=0x7fe164148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe1637fe640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1640b00f0 0x7fe164148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe1637fe640 1 -- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1640b2850 msgr2=0x7fe164145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe1637fe640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1640b2850 0x7fe164145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe1637fe640 1 -- 172.21.15.46:0/544233253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1641e24a0 con 0x7fe150003980 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe163fff640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1640b2850 0x7fe164145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe15bfff640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1640b00f0 0x7fe164148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe1637fe640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe1641e3000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe15400c190 tx=0x7fe150005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe160ff9640 1 -- 172.21.15.46:0/544233253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154001680 con 0x7fe150003980 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe160ff9640 1 -- 172.21.15.46:0/544233253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe154004a00 con 0x7fe150003980 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1640b0950 con 0x7fe150003980 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe160ff9640 1 -- 172.21.15.46:0/544233253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154004d70 con 0x7fe150003980 2023-12-16T21:43:03.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1640b6e30 con 0x7fe150003980 2023-12-16T21:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe164068960 con 0x7fe150003980 2023-12-16T21:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe160ff9640 1 -- 172.21.15.46:0/544233253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fe154007090 con 0x7fe150003980 2023-12-16T21:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe160ff9640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe160ff9640 1 -- 172.21.15.46:0/544233253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe15411d870 con 0x7fe150003980 2023-12-16T21:43:03.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe15bfff640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.559+0000 7fe15bfff640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe1440664a0 0x7fe144068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe164144510 tx=0x7fe14c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.563+0000 7fe160ff9640 1 -- 172.21.15.46:0/544233253 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fe1540ead60 con 0x7fe150003980 2023-12-16T21:43:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.719+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1640b2470 con 0x7fe150003980 2023-12-16T21:43:03.726 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.719+0000 7fe160ff9640 1 -- 172.21.15.46:0/544233253 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1540ee380 con 0x7fe150003980 2023-12-16T21:43:03.727 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:43:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.719+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe1440664a0 msgr2=0x7fe144068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.719+0000 7fe169c55640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe1440664a0 0x7fe144068960 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe164144510 tx=0x7fe14c008040 comp rx=0 tx=0).stop 2023-12-16T21:43:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.719+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 msgr2=0x7fe1641e3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.719+0000 7fe169c55640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe1641e3000 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe15400c190 tx=0x7fe150005800 comp rx=0 tx=0).stop 2023-12-16T21:43:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 shutdown_connections 2023-12-16T21:43:03.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe1440664a0 0x7fe144068960 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe1640b2850 0x7fe164145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe1640b00f0 0x7fe164148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 --2- 172.21.15.46:0/544233253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe150003980 0x7fe1641e3000 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 >> 172.21.15.46:0/544233253 conn(0x7fe16406b860 msgr2=0x7fe164146f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 shutdown_connections 2023-12-16T21:43:03.730 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.723+0000 7fe169c55640 1 -- 172.21.15.46:0/544233253 wait complete. 2023-12-16T21:43:03.747 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:43:03.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb12dae640 1 Processor -- start 2023-12-16T21:43:03.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb12dae640 1 -- start start 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb12dae640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdb0c0b1eb0 0x7fdb0c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb12dae640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdb0c0af300 0x7fdb0c144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb12dae640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0c0b1730 con 0x7fdb0c0b0470 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb12dae640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c06cd70 con 0x7fdb0c0af300 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb12dae640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0c0b0ad0 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb0bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdb0c0af300 0x7fdb0c144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb0bfff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdb0c0af300 0x7fdb0c144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49564/0 (socket says 172.21.15.46:49564) 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb0bfff640 1 -- 172.21.15.46:0/4175203946 learned_addr learned my addr 172.21.15.46:0/4175203946 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb10b4a640 1 --2- 172.21.15.46:0/4175203946 >> v2:172.21.15.161:3300/0 conn(0x7fdb0c0b1eb0 0x7fdb0c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb0bfff640 1 -- 172.21.15.46:0/4175203946 >> v2:172.21.15.161:3300/0 conn(0x7fdb0c0b1eb0 msgr2=0x7fdb0c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb0bfff640 1 --2- 172.21.15.46:0/4175203946 >> v2:172.21.15.161:3300/0 conn(0x7fdb0c0b1eb0 0x7fdb0c0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb0bfff640 1 -- 172.21.15.46:0/4175203946 >> v1:172.21.15.161:6789/0 conn(0x7fdb0c0b0470 legacy=0x7fdb0c1479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.871+0000 7fdb0bfff640 1 -- 172.21.15.46:0/4175203946 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c06c7f0 con 0x7fdb0c0af300 2023-12-16T21:43:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb0bfff640 1 --2- 172.21.15.46:0/4175203946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0af300 0x7fdb0c144ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0af030 tx=0x7fdafc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168058c2ae85ca1a server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb0b7fe640 1 -- 172.21.15.46:0/4175203946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0ac640 con 0x7fdb0c0af300 2023-12-16T21:43:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb0b7fe640 1 -- 172.21.15.46:0/4175203946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc0ac7a0 con 0x7fdb0c0af300 2023-12-16T21:43:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb0b7fe640 1 -- 172.21.15.46:0/4175203946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0acb30 con 0x7fdb0c0af300 2023-12-16T21:43:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- 172.21.15.46:0/4175203946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0af300 msgr2=0x7fdb0c144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 --2- 172.21.15.46:0/4175203946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0af300 0x7fdb0c144ab0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0af030 tx=0x7fdafc01c3e0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- 172.21.15.46:0/4175203946 shutdown_connections 2023-12-16T21:43:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 --2- 172.21.15.46:0/4175203946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0af300 0x7fdb0c144ab0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 --2- 172.21.15.46:0/4175203946 >> v2:172.21.15.161:3300/0 conn(0x7fdb0c0b1eb0 0x7fdb0c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- 172.21.15.46:0/4175203946 >> 172.21.15.46:0/4175203946 conn(0x7fdb0c0b4900 msgr2=0x7fdb0c0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- 172.21.15.46:0/4175203946 shutdown_connections 2023-12-16T21:43:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- 172.21.15.46:0/4175203946 wait complete. 2023-12-16T21:43:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 Processor -- start 2023-12-16T21:43:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- start start 2023-12-16T21:43:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb0c0af300 0x7fdb0c13e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb0c0b0470 0x7fdb0c09a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0b1eb0 0x7fdb0c1e0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c1e0620 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c099ed0 con 0x7fdb0c0af300 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb12dae640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0c09a970 con 0x7fdb0c0b0470 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0b1eb0 0x7fdb0c1e0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0b1eb0 0x7fdb0c1e0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49578/0 (socket says 172.21.15.46:49578) 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 -- 172.21.15.46:0/1198439457 learned_addr learned my addr 172.21.15.46:0/1198439457 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb10b4a640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb0c0af300 0x7fdb0c13e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb0bfff640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb0c0b0470 0x7fdb0c09a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 -- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb0c0b0470 msgr2=0x7fdb0c09a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb0c0b0470 0x7fdb0c09a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 -- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb0c0af300 msgr2=0x7fdb0c13e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb0c0af300 0x7fdb0c13e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb1134b640 1 -- 172.21.15.46:0/1198439457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc0094d0 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.875+0000 7fdb0bfff640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb0c0b0470 0x7fdb0c09a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb1134b640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0b1eb0 0x7fdb0c1e0d20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c6f0 tx=0x7fdb0400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb097fa640 1 -- 172.21.15.46:0/1198439457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04003980 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb097fa640 1 -- 172.21.15.46:0/1198439457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb04003ae0 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c1dd0b0 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb097fa640 1 -- 172.21.15.46:0/1198439457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb04003e90 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c099bf0 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0c068960 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb097fa640 1 -- 172.21.15.46:0/1198439457 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fdb0400a7f0 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb097fa640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdaec066470 0x7fdaec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb097fa640 1 -- 172.21.15.46:0/1198439457 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb04083b50 con 0x7fdb0c0b1eb0 2023-12-16T21:43:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb10b4a640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdaec066470 0x7fdaec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:03.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.879+0000 7fdb10b4a640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdaec066470 0x7fdaec068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009a50 tx=0x7fdaf4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:03.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:03.883+0000 7fdb097fa640 1 -- 172.21.15.46:0/1198439457 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fdb04051060 con 0x7fdb0c0b1eb0 2023-12-16T21:43:04.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdb0c0b62c0 con 0x7fdb0c0b1eb0 2023-12-16T21:43:04.045 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb097fa640 1 -- 172.21.15.46:0/1198439457 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdb04054680 con 0x7fdb0c0b1eb0 2023-12-16T21:43:04.045 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:43:04.047 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdaec066470 msgr2=0x7fdaec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdaec066470 0x7fdaec068930 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009a50 tx=0x7fdaf4008040 comp rx=0 tx=0).stop 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0b1eb0 msgr2=0x7fdb0c1e0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0b1eb0 0x7fdb0c1e0d20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400c6f0 tx=0x7fdb0400cbb0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 shutdown_connections 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdaec066470 0x7fdaec068930 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb0c0b1eb0 0x7fdb0c1e0d20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdb0c0b0470 0x7fdb0c09a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 --2- 172.21.15.46:0/1198439457 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdb0c0af300 0x7fdb0c13e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 >> 172.21.15.46:0/1198439457 conn(0x7fdb0c0b4900 msgr2=0x7fdb0c0b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 shutdown_connections 2023-12-16T21:43:04.048 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.039+0000 7fdb12dae640 1 -- 172.21.15.46:0/1198439457 wait complete. 2023-12-16T21:43:04.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:43:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.183+0000 7f5708c2c640 1 Processor -- start 2023-12-16T21:43:04.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.183+0000 7f5708c2c640 1 -- start start 2023-12-16T21:43:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57040b1eb0 0x7f57040aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57040b1730 con 0x7f57040af300 2023-12-16T21:43:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570406c7f0 con 0x7f57041451c0 2023-12-16T21:43:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570406cd70 con 0x7f57040b1eb0 2023-12-16T21:43:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570259c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f57040b1eb0 0x7f57040aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5701d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f57040af300 0x7f5704144ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55212/0 (socket says 172.21.15.46:55212) 2023-12-16T21:43:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5701d9b640 1 -- 172.21.15.46:0/4282767543 learned_addr learned my addr 172.21.15.46:0/4282767543 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570259c640 1 -- 172.21.15.46:0/4282767543 >> v1:172.21.15.161:6789/0 conn(0x7f57041451c0 legacy=0x7f570414ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570259c640 1 -- 172.21.15.46:0/4282767543 >> v1:172.21.15.46:6789/0 conn(0x7f57040af300 legacy=0x7f5704144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.194 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570259c640 1 -- 172.21.15.46:0/4282767543 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040b2290 con 0x7f57040b1eb0 2023-12-16T21:43:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570259c640 1 --2- 172.21.15.46:0/4282767543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57040b1eb0 0x7f57040aff30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f56f80097c0 tx=0x7f56f801c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3148ad4b6fb0702d server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570159a640 1 -- 172.21.15.46:0/4282767543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80ab600 con 0x7f57040b1eb0 2023-12-16T21:43:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570159a640 1 -- 172.21.15.46:0/4282767543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f80a7050 con 0x7f57040b1eb0 2023-12-16T21:43:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f570159a640 1 -- 172.21.15.46:0/4282767543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80ab980 con 0x7f57040b1eb0 2023-12-16T21:43:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- 172.21.15.46:0/4282767543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57040b1eb0 msgr2=0x7f57040aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 --2- 172.21.15.46:0/4282767543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57040b1eb0 0x7f57040aff30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f56f80097c0 tx=0x7f56f801c320 comp rx=0 tx=0).stop 2023-12-16T21:43:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- 172.21.15.46:0/4282767543 shutdown_connections 2023-12-16T21:43:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 --2- 172.21.15.46:0/4282767543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57040b1eb0 0x7f57040aff30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- 172.21.15.46:0/4282767543 >> 172.21.15.46:0/4282767543 conn(0x7f57040b4900 msgr2=0x7f57040b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- 172.21.15.46:0/4282767543 shutdown_connections 2023-12-16T21:43:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.187+0000 7f5708c2c640 1 -- 172.21.15.46:0/4282767543 wait complete. 2023-12-16T21:43:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 Processor -- start 2023-12-16T21:43:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 -- start start 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57040af300 0x7f570406ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57040b1eb0 0x7f5704137210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57041451c0 0x7f570406ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040ae6d0 con 0x7f57041451c0 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040704e0 con 0x7f57040af300 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570414f760 con 0x7f57040b1eb0 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57041451c0 0x7f570406ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57041451c0 0x7f570406ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49590/0 (socket says 172.21.15.46:49590) 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 -- 172.21.15.46:0/2036093364 learned_addr learned my addr 172.21.15.46:0/2036093364 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5701d9b640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57040b1eb0 0x7f5704137210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f570259c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57040af300 0x7f570406ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 -- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57040b1eb0 msgr2=0x7f5704137210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57040b1eb0 0x7f5704137210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 -- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57040af300 msgr2=0x7f570406ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57040af300 0x7f570406ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 -- 172.21.15.46:0/2036093364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f8009420 con 0x7f57041451c0 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5701d9b640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57040b1eb0 0x7f5704137210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f570259c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57040af300 0x7f570406ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5702d9d640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57041451c0 0x7f570406ea70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00ba10 tx=0x7f56fc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f56f2ffd640 1 -- 172.21.15.46:0/2036093364 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00cac0 con 0x7f57041451c0 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f56f2ffd640 1 -- 172.21.15.46:0/2036093364 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc00cc20 con 0x7f57041451c0 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f56f2ffd640 1 -- 172.21.15.46:0/2036093364 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc014440 con 0x7f57041451c0 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57040707c0 con 0x7f57041451c0 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5704137f70 con 0x7f57041451c0 2023-12-16T21:43:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.191+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57040ae880 con 0x7f57041451c0 2023-12-16T21:43:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.195+0000 7f56f2ffd640 1 -- 172.21.15.46:0/2036093364 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f56fc01d3b0 con 0x7f57041451c0 2023-12-16T21:43:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.195+0000 7f56f2ffd640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f56e4066470 0x7f56e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.195+0000 7f56f2ffd640 1 -- 172.21.15.46:0/2036093364 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56fc083c70 con 0x7f57041451c0 2023-12-16T21:43:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.195+0000 7f570259c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f56e4066470 0x7f56e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.195+0000 7f570259c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f56e4066470 0x7f56e4068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f56f80ad000 tx=0x7f56f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.199+0000 7f56f2ffd640 1 -- 172.21.15.46:0/2036093364 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f56fc051180 con 0x7f57041451c0 2023-12-16T21:43:04.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.351+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5704068940 con 0x7f57041451c0 2023-12-16T21:43:04.361 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f56f2ffd640 1 -- 172.21.15.46:0/2036093364 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f56fc0547a0 con 0x7f57041451c0 2023-12-16T21:43:04.362 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f56e4066470 msgr2=0x7f56e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f56e4066470 0x7f56e4068930 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f56f80ad000 tx=0x7f56f8008040 comp rx=0 tx=0).stop 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57041451c0 msgr2=0x7f570406ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57041451c0 0x7f570406ea70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f56fc00ba10 tx=0x7f56fc00bed0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 shutdown_connections 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f56e4066470 0x7f56e4068930 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f57041451c0 0x7f570406ea70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f57040b1eb0 0x7f5704137210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 --2- 172.21.15.46:0/2036093364 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f57040af300 0x7f570406ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 >> 172.21.15.46:0/2036093364 conn(0x7f57040b4900 msgr2=0x7f57040b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 shutdown_connections 2023-12-16T21:43:04.364 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.355+0000 7f5708c2c640 1 -- 172.21.15.46:0/2036093364 wait complete. 2023-12-16T21:43:04.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-16T21:43:04.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f1673238640 1 Processor -- start 2023-12-16T21:43:04.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f1673238640 1 -- start start 2023-12-16T21:43:04.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f1673238640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c0b1f80 0x7f166c14e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f1673238640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c0af330 con 0x7f166c0adf50 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f1673238640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f166c0b2ea0 con 0x7f166c06c1a0 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f1673238640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0aed20 con 0x7f166c0b1f80 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f166c0adf50 0x7f166c0b1870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55218/0 (socket says 172.21.15.46:55218) 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166bfff640 1 -- 172.21.15.46:0/4255763636 learned_addr learned my addr 172.21.15.46:0/4255763636 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f16717d5640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c0b1f80 0x7f166c14e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610790094 0 0) 0x7f166c0af330 con 0x7f166c0adf50 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1654003530 con 0x7f166c0adf50 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1579565243 0 0) 0x7f1654003530 con 0x7f166c0adf50 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c0b1f80 msgr2=0x7f166c14e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:04.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c0b1f80 0x7f166c14e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 >> v1:172.21.15.161:6789/0 conn(0x7f166c06c1a0 legacy=0x7f166c0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c06c580 con 0x7f166c0adf50 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f1660001680 con 0x7f166c0adf50 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 >> v1:172.21.15.46:6789/0 conn(0x7f166c0adf50 legacy=0x7f166c0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16540039b0 0x7f1654003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c0b1f80 0x7f16540047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654004d10 0x7f1654005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c0aed20 con 0x7f1654004d10 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16540099b0 con 0x7f16540039b0 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1654009b70 con 0x7f166c0b1f80 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f16717d5640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654004d10 0x7f1654005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f1670fd4640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16540039b0 0x7f1654003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.505 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f16717d5640 1 -- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c0b1f80 msgr2=0x7f16540047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f16717d5640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f166c0b1f80 0x7f16540047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f16717d5640 1 -- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16540039b0 msgr2=0x7f1654003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f16717d5640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f16540039b0 0x7f1654003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.495+0000 7f16717d5640 1 -- 172.21.15.46:0/4255763636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1654003530 con 0x7f1654004d10 2023-12-16T21:43:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f16717d5640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654004d10 0x7f1654005190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1664009fc0 tx=0x7f166401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea57c99c298edd9b server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640a5cb0 con 0x7f1654004d10 2023-12-16T21:43:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f166400f040 con 0x7f1654004d10 2023-12-16T21:43:04.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f166b7fe640 1 -- 172.21.15.46:0/4255763636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640ad680 con 0x7f1654004d10 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- 172.21.15.46:0/4255763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654004d10 msgr2=0x7f1654005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 --2- 172.21.15.46:0/4255763636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1654004d10 0x7f1654005190 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1664009fc0 tx=0x7f166401cca0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f16717d5640 1 -- 172.21.15.46:0/4255763636 reap_dead start 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- 172.21.15.46:0/4255763636 shutdown_connections 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- 172.21.15.46:0/4255763636 >> 172.21.15.46:0/4255763636 conn(0x7f166c06a580 msgr2=0x7f166c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- 172.21.15.46:0/4255763636 shutdown_connections 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- 172.21.15.46:0/4255763636 wait complete. 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 Processor -- start 2023-12-16T21:43:04.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- start start 2023-12-16T21:43:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1654004d10 0x7f166c09ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c1dbe10 0x7f166c1d5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f166c1dc290 0x7f166c1d57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c1c8d00 con 0x7f166c1dc290 2023-12-16T21:43:04.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166c1db6a0 con 0x7f166c1dbe10 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1673238640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166c13dd00 con 0x7f1654004d10 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f16717d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f166c1dc290 0x7f166c1d57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f16717d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f166c1dc290 0x7f166c1d57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49614/0 (socket says 172.21.15.46:49614) 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f16717d5640 1 -- 172.21.15.46:0/2263506968 learned_addr learned my addr 172.21.15.46:0/2263506968 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f1670fd4640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1654004d10 0x7f166c09ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.499+0000 7f166bfff640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c1dbe10 0x7f166c1d5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16717d5640 1 -- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1654004d10 msgr2=0x7f166c09ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16717d5640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1654004d10 0x7f166c09ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16717d5640 1 -- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c1dbe10 msgr2=0x7f166c1d5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16717d5640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c1dbe10 0x7f166c1d5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16717d5640 1 -- 172.21.15.46:0/2263506968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c1d5d00 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f166bfff640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c1dbe10 0x7f166c1d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f1670fd4640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1654004d10 0x7f166c09ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16717d5640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f166c1dc290 0x7f166c1d57c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16640b0040 tx=0x7f1664009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16697fa640 1 -- 172.21.15.46:0/2263506968 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16640039a0 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16697fa640 1 -- 172.21.15.46:0/2263506968 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1664003b00 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f166c0718d0 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16697fa640 1 -- 172.21.15.46:0/2263506968 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1664003e70 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f166c1c9980 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f166c0679e0 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16697fa640 1 -- 172.21.15.46:0/2263506968 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f16640b4040 con 0x7f166c1dc290 2023-12-16T21:43:04.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16697fa640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f16580664a0 0x7f1658068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f16697fa640 1 -- 172.21.15.46:0/2263506968 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f166411d840 con 0x7f166c1dc290 2023-12-16T21:43:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f1670fd4640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f16580664a0 0x7f1658068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.503+0000 7f1670fd4640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f16580664a0 0x7f1658068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f165c0016a0 tx=0x7f165c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.507+0000 7f16697fa640 1 -- 172.21.15.46:0/2263506968 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f16640ead70 con 0x7f166c1dc290 2023-12-16T21:43:04.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.679+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f166c071b10 con 0x7f166c1dc290 2023-12-16T21:43:04.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.679+0000 7f16697fa640 1 -- 172.21.15.46:0/2263506968 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f16640b2080 con 0x7f166c1dc290 2023-12-16T21:43:04.687 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-16T21:43:04.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f16580664a0 msgr2=0x7f1658068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f16580664a0 0x7f1658068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f165c0016a0 tx=0x7f165c009040 comp rx=0 tx=0).stop 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f166c1dc290 msgr2=0x7f166c1d57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f166c1dc290 0x7f166c1d57c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f16640b0040 tx=0x7f1664009750 comp rx=0 tx=0).stop 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 shutdown_connections 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f16580664a0 0x7f1658068960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f166c1dc290 0x7f166c1d57c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f166c1dbe10 0x7f166c1d5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 --2- 172.21.15.46:0/2263506968 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1654004d10 0x7f166c09ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 >> 172.21.15.46:0/2263506968 conn(0x7f166c06a580 msgr2=0x7f166c0b2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 shutdown_connections 2023-12-16T21:43:04.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.683+0000 7f1673238640 1 -- 172.21.15.46:0/2263506968 wait complete. 2023-12-16T21:43:04.707 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:43:04.707 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-16T21:43:04.712 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.871+0000 7f2c4e3b7640 1 Processor -- start 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.871+0000 7f2c4e3b7640 1 -- start start 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c4e3b7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 0x7f2c481520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c4e3b7640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c480b4530 con 0x7f2c480b5de0 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c4e3b7640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c480b61c0 con 0x7f2c480b54c0 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c4e3b7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c480b4780 con 0x7f2c480716e0 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2c480b5de0 0x7f2c4806fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55232/0 (socket says 172.21.15.46:55232) 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 -- 172.21.15.46:0/991269831 learned_addr learned my addr 172.21.15.46:0/991269831 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:04.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c4c954640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 0x7f2c481520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998128160 0 0) 0x7f2c480b4530 con 0x7f2c480b5de0 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c30003530 con 0x7f2c480b5de0 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1763241753 0 0) 0x7f2c30003530 con 0x7f2c480b5de0 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 msgr2=0x7f2c481520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 0x7f2c481520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 >> v1:172.21.15.161:6789/0 conn(0x7f2c480b54c0 legacy=0x7f2c48140630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c480afb30 con 0x7f2c480b5de0 2023-12-16T21:43:04.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f2c38001680 con 0x7f2c480b5de0 2023-12-16T21:43:04.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 >> v1:172.21.15.46:6789/0 conn(0x7f2c480b5de0 legacy=0x7f2c4806fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c300047d0 0x7f2c30004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 0x7f2c30005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c480b4780 con 0x7f2c300039b0 2023-12-16T21:43:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3000ba40 con 0x7f2c300047d0 2023-12-16T21:43:04.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3000bc00 con 0x7f2c480716e0 2023-12-16T21:43:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c477fe640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c300047d0 0x7f2c30004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c4c954640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 0x7f2c30005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 -- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 msgr2=0x7f2c30005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 0x7f2c30005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 -- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c300047d0 msgr2=0x7f2c30004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c300047d0 0x7f2c30004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c47fff640 1 -- 172.21.15.46:0/991269831 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c30003530 con 0x7f2c300039b0 2023-12-16T21:43:04.891 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.875+0000 7f2c4c954640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c480716e0 0x7f2c30005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c30003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c38001060 tx=0x7f2c3801cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25daff1b01d938ef server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c380058e0 con 0x7f2c300039b0 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c38005a60 con 0x7f2c300039b0 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c46ffd640 1 -- 172.21.15.46:0/991269831 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38005e10 con 0x7f2c300039b0 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/991269831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 msgr2=0x7f2c30003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 --2- 172.21.15.46:0/991269831 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c30003dd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2c38001060 tx=0x7f2c3801cb70 comp rx=0 tx=0).stop 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4c954640 1 -- 172.21.15.46:0/991269831 reap_dead start 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/991269831 shutdown_connections 2023-12-16T21:43:04.896 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/991269831 >> 172.21.15.46:0/991269831 conn(0x7f2c480b9250 msgr2=0x7f2c480b9660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/991269831 shutdown_connections 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/991269831 wait complete. 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 Processor -- start 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- start start 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c4813f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c4813f580 0x7f2c480ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c480aeaf0 0x7f2c481debd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c481e2d40 con 0x7f2c300039b0 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48136d20 con 0x7f2c480aeaf0 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c48144200 con 0x7f2c4813f580 2023-12-16T21:43:04.897 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c4813f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c4813f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49630/0 (socket says 172.21.15.46:49630) 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 -- 172.21.15.46:0/3652636269 learned_addr learned my addr 172.21.15.46:0/3652636269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4c954640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c480aeaf0 0x7f2c481debd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c477fe640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c4813f580 0x7f2c480ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 -- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c4813f580 msgr2=0x7f2c480ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c4813f580 0x7f2c480ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 -- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c480aeaf0 msgr2=0x7f2c481debd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c480aeaf0 0x7f2c481debd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 -- 172.21.15.46:0/3652636269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c481e2680 con 0x7f2c300039b0 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c47fff640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c4813f040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2c380ab040 tx=0x7f2c30005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c44ff9640 1 -- 172.21.15.46:0/3652636269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c38005e90 con 0x7f2c300039b0 2023-12-16T21:43:04.898 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c44ff9640 1 -- 172.21.15.46:0/3652636269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c380a5810 con 0x7f2c300039b0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c44ff9640 1 -- 172.21.15.46:0/3652636269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c380a5ba0 con 0x7f2c300039b0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.879+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c481e2960 con 0x7f2c300039b0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.883+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c481e2350 con 0x7f2c300039b0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.883+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c481dea20 con 0x7f2c300039b0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.883+0000 7f2c44ff9640 1 -- 172.21.15.46:0/3652636269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2c380a5d00 con 0x7f2c300039b0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.883+0000 7f2c44ff9640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2c340664a0 0x7f2c34068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.883+0000 7f2c44ff9640 1 -- 172.21.15.46:0/3652636269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c3811da00 con 0x7f2c300039b0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.883+0000 7f2c477fe640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2c340664a0 0x7f2c34068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.883+0000 7f2c477fe640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2c340664a0 0x7f2c34068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c481401a0 tx=0x7f2c3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:04.899 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:04.887+0000 7f2c44ff9640 1 -- 172.21.15.46:0/3652636269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2c380ee9a0 con 0x7f2c300039b0 2023-12-16T21:43:05.051 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.043+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c480688f0 con 0x7f2c300039b0 2023-12-16T21:43:05.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c44ff9640 1 -- 172.21.15.46:0/3652636269 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f2c380ee530 con 0x7f2c300039b0 2023-12-16T21:43:05.054 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:05.054 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2c340664a0 msgr2=0x7f2c34068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2c340664a0 0x7f2c34068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c481401a0 tx=0x7f2c3c009040 comp rx=0 tx=0).stop 2023-12-16T21:43:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 msgr2=0x7f2c4813f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.056 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c4813f040 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2c380ab040 tx=0x7f2c30005800 comp rx=0 tx=0).stop 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 shutdown_connections 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2c340664a0 0x7f2c34068960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2c480aeaf0 0x7f2c481debd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2c4813f580 0x7f2c480ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 --2- 172.21.15.46:0/3652636269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2c300039b0 0x7f2c4813f040 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 >> 172.21.15.46:0/3652636269 conn(0x7f2c480b9250 msgr2=0x7f2c48139f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 shutdown_connections 2023-12-16T21:43:05.057 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.047+0000 7f2c4e3b7640 1 -- 172.21.15.46:0/3652636269 wait complete. 2023-12-16T21:43:05.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:43:05.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:43:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfedca640 1 Processor -- start 2023-12-16T21:43:05.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfedca640 1 -- start start 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfedca640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f3cf8073710 0x7f3cf80ae770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfedca640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf8073af0 con 0x7f3cf8143e20 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfedca640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cf80af540 con 0x7f3cf80ac8f0 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfedca640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cf80722a0 con 0x7f3cf8073710 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfd367640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f3cf8143e20 0x7f3cf8150ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55234/0 (socket says 172.21.15.46:55234) 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfd367640 1 -- 172.21.15.46:0/3067903351 learned_addr learned my addr 172.21.15.46:0/3067903351 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.195+0000 7f3cfcb66640 1 --2- 172.21.15.46:0/3067903351 >> v2:172.21.15.161:3300/0 conn(0x7f3cf8073710 0x7f3cf80ae770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605109322 0 0) 0x7f3cf8073af0 con 0x7f3cf8143e20 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce4003530 con 0x7f3cf8143e20 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922740214 0 0) 0x7f3ce4003530 con 0x7f3cf8143e20 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 >> v2:172.21.15.161:3300/0 conn(0x7f3cf8073710 msgr2=0x7f3cf80ae770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 --2- 172.21.15.46:0/3067903351 >> v2:172.21.15.161:3300/0 conn(0x7f3cf8073710 0x7f3cf80ae770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 >> v1:172.21.15.161:6789/0 conn(0x7f3cf80ac8f0 legacy=0x7f3cf8143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf80aedd0 con 0x7f3cf8143e20 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f3cf0001680 con 0x7f3cf8143e20 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 >> v1:172.21.15.46:6789/0 conn(0x7f3cf8143e20 legacy=0x7f3cf8150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce4003980 0x7f3ce4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ce40047d0 0x7f3ce4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3ce400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf80722a0 con 0x7f3ce4005190 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce400c130 con 0x7f3ce4003980 2023-12-16T21:43:05.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce400c2f0 con 0x7f3ce40047d0 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3ce400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfcb66640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce4003980 0x7f3ce4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf7fff640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ce40047d0 0x7f3ce4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 -- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ce40047d0 msgr2=0x7f3ce4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3ce40047d0 0x7f3ce4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 -- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce4003980 msgr2=0x7f3ce4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3ce4003980 0x7f3ce4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 -- 172.21.15.46:0/3067903351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce4003530 con 0x7f3ce4005190 2023-12-16T21:43:05.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 -- 172.21.15.46:0/3067903351 reap_dead start 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfd367640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3ce400b990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000c240 tx=0x7f3cf001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2ce861f3e71bff server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0005ae0 con 0x7f3ce4005190 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf00ab070 con 0x7f3ce4005190 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cf77fe640 1 -- 172.21.15.46:0/3067903351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf0007070 con 0x7f3ce4005190 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfedca640 1 -- 172.21.15.46:0/3067903351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 msgr2=0x7f3ce400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3ce400b990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000c240 tx=0x7f3cf001f3b0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfedca640 1 -- 172.21.15.46:0/3067903351 shutdown_connections 2023-12-16T21:43:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3067903351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3ce400b990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfedca640 1 -- 172.21.15.46:0/3067903351 >> 172.21.15.46:0/3067903351 conn(0x7f3cf806a580 msgr2=0x7f3cf806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.199+0000 7f3cfedca640 1 -- 172.21.15.46:0/3067903351 shutdown_connections 2023-12-16T21:43:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- 172.21.15.46:0/3067903351 wait complete. 2023-12-16T21:43:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 Processor -- start 2023-12-16T21:43:05.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- start start 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3cf81df470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cf813af20 0x7f3cf813b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cf8136660 0x7f3cf8136ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf813bf60 con 0x7f3ce4005190 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81df9b0 con 0x7f3cf813af20 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf81dff60 con 0x7f3cf8136660 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3cf81df470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3cf81df470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49634/0 (socket says 172.21.15.46:49634) 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 -- 172.21.15.46:0/3616187606 learned_addr learned my addr 172.21.15.46:0/3616187606 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cf7fff640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cf813af20 0x7f3cf813b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfd367640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cf8136660 0x7f3cf8136ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 -- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cf8136660 msgr2=0x7f3cf8136ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cf8136660 0x7f3cf8136ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 -- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cf813af20 msgr2=0x7f3cf813b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cf813af20 0x7f3cf813b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 -- 172.21.15.46:0/3616187606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf813b8e0 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cf7fff640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cf813af20 0x7f3cf813b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfcb66640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3cf81df470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009fb0 tx=0x7f3ce80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cf57fa640 1 -- 172.21.15.46:0/3616187606 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800e030 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cf57fa640 1 -- 172.21.15.46:0/3616187606 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce8016470 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf813cea0 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cf57fa640 1 -- 172.21.15.46:0/3616187606 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce8016820 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf813ab60 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.203+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf806bd00 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.207+0000 7f3cf57fa640 1 -- 172.21.15.46:0/3616187606 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f3ce8016980 con 0x7f3ce4005190 2023-12-16T21:43:05.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.207+0000 7f3cf57fa640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3cd80664a0 0x7f3cd8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.207+0000 7f3cf57fa640 1 -- 172.21.15.46:0/3616187606 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ce800f070 con 0x7f3ce4005190 2023-12-16T21:43:05.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.207+0000 7f3cf7fff640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3cd80664a0 0x7f3cd8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.207+0000 7f3cf7fff640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3cd80664a0 0x7f3cd8068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0000c10 tx=0x7f3ce0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.217 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.211+0000 7f3cf57fa640 1 -- 172.21.15.46:0/3616187606 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f3ce80521b0 con 0x7f3ce4005190 2023-12-16T21:43:05.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.371+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cf8068960 con 0x7f3ce4005190 2023-12-16T21:43:05.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.371+0000 7f3cf57fa640 1 -- 172.21.15.46:0/3616187606 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3ce80557d0 con 0x7f3ce4005190 2023-12-16T21:43:05.381 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:43:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3cd80664a0 msgr2=0x7f3cd8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3cd80664a0 0x7f3cd8068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ce0000c10 tx=0x7f3ce0009040 comp rx=0 tx=0).stop 2023-12-16T21:43:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 msgr2=0x7f3cf81df470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3cf81df470 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3ce8009fb0 tx=0x7f3ce80115a0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 shutdown_connections 2023-12-16T21:43:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f3cd80664a0 0x7f3cd8068960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f3cf8136660 0x7f3cf8136ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3cf813af20 0x7f3cf813b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 --2- 172.21.15.46:0/3616187606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3ce4005190 0x7f3cf81df470 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 >> 172.21.15.46:0/3616187606 conn(0x7f3cf806a580 msgr2=0x7f3cf806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 shutdown_connections 2023-12-16T21:43:05.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.375+0000 7f3cfedca640 1 -- 172.21.15.46:0/3616187606 wait complete. 2023-12-16T21:43:05.400 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:43:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897e62c640 1 Processor -- start 2023-12-16T21:43:05.529 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897e62c640 1 -- start start 2023-12-16T21:43:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897e62c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f897806c8b0 0x7f89780b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897e62c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f89780ae750 0x7f89780ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897e62c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89780b02e0 con 0x7f89780af300 2023-12-16T21:43:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897e62c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89780aded0 con 0x7f897806c8b0 2023-12-16T21:43:05.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897e62c640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897806cc90 con 0x7f89780ae750 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f8977fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f897806c8b0 0x7f89780b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f896f7fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f89780af300 0x7f89780ace20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55236/0 (socket says 172.21.15.46:55236) 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f896f7fe640 1 -- 172.21.15.46:0/2443853103 learned_addr learned my addr 172.21.15.46:0/2443853103 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897cbc9640 1 --2- 172.21.15.46:0/2443853103 >> v2:172.21.15.161:3300/0 conn(0x7f89780ae750 0x7f89780ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f8977fff640 1 -- 172.21.15.46:0/2443853103 >> v2:172.21.15.161:3300/0 conn(0x7f89780ae750 msgr2=0x7f89780ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f8977fff640 1 --2- 172.21.15.46:0/2443853103 >> v2:172.21.15.161:3300/0 conn(0x7f89780ae750 0x7f89780ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f8977fff640 1 -- 172.21.15.46:0/2443853103 >> v1:172.21.15.46:6789/0 conn(0x7f89780af300 legacy=0x7f89780ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f89777fe640 1 -- 172.21.15.46:0/2443853103 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927455411 0 0) 0x7f89780b02e0 con 0x7f89780af300 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f8977fff640 1 -- 172.21.15.46:0/2443853103 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780b05c0 con 0x7f897806c8b0 2023-12-16T21:43:05.531 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.523+0000 7f897cbc9640 1 --2- 172.21.15.46:0/2443853103 >> v2:172.21.15.161:3300/0 conn(0x7f89780ae750 0x7f89780ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 --2- 172.21.15.46:0/2443853103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f89780b4220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f897809db60 tx=0x7f896801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335e395e9098a8ce server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f89777fe640 1 -- 172.21.15.46:0/2443853103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680ab600 con 0x7f897806c8b0 2023-12-16T21:43:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f89777fe640 1 -- 172.21.15.46:0/2443853103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89680a7050 con 0x7f897806c8b0 2023-12-16T21:43:05.533 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f89777fe640 1 -- 172.21.15.46:0/2443853103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680ab980 con 0x7f897806c8b0 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- 172.21.15.46:0/2443853103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 msgr2=0x7f89780b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 --2- 172.21.15.46:0/2443853103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f89780b4220 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f897809db60 tx=0x7f896801c3a0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- 172.21.15.46:0/2443853103 shutdown_connections 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 --2- 172.21.15.46:0/2443853103 >> v2:172.21.15.161:3300/0 conn(0x7f89780ae750 0x7f89780ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 --2- 172.21.15.46:0/2443853103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f89780b4220 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- 172.21.15.46:0/2443853103 >> 172.21.15.46:0/2443853103 conn(0x7f89780b4900 msgr2=0x7f89780b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- 172.21.15.46:0/2443853103 shutdown_connections 2023-12-16T21:43:05.534 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- 172.21.15.46:0/2443853103 wait complete. 2023-12-16T21:43:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 Processor -- start 2023-12-16T21:43:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- start start 2023-12-16T21:43:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f8978141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89780ae750 0x7f89781dfc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89780af300 0x7f89781e01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89781409b0 con 0x7f897806c8b0 2023-12-16T21:43:05.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89781411f0 con 0x7f89780af300 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8978140260 con 0x7f89780ae750 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f8978141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f8978141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49650/0 (socket says 172.21.15.46:49650) 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 -- 172.21.15.46:0/2707832932 learned_addr learned my addr 172.21.15.46:0/2707832932 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897cbc9640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89780af300 0x7f89781e01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f896f7fe640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89780ae750 0x7f89781dfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 -- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89780ae750 msgr2=0x7f89781dfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89780ae750 0x7f89781dfc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 -- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89780af300 msgr2=0x7f89781e01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89780af300 0x7f89781e01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 -- 172.21.15.46:0/2707832932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8968009420 con 0x7f897806c8b0 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897cbc9640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89780af300 0x7f89781e01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f8977fff640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f8978141a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f89680ae040 tx=0x7f8968019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.537 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f89757fa640 1 -- 172.21.15.46:0/2707832932 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680b4610 con 0x7f897806c8b0 2023-12-16T21:43:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f89757fa640 1 -- 172.21.15.46:0/2707832932 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89680abe90 con 0x7f897806c8b0 2023-12-16T21:43:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8978140ba0 con 0x7f897806c8b0 2023-12-16T21:43:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f89757fa640 1 -- 172.21.15.46:0/2707832932 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89680b4970 con 0x7f897806c8b0 2023-12-16T21:43:05.538 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.527+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897813dae0 con 0x7f897806c8b0 2023-12-16T21:43:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.531+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8978068960 con 0x7f897806c8b0 2023-12-16T21:43:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.531+0000 7f89757fa640 1 -- 172.21.15.46:0/2707832932 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f89680b2040 con 0x7f897806c8b0 2023-12-16T21:43:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.531+0000 7f89757fa640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8958066470 0x7f8958068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.531+0000 7f89757fa640 1 -- 172.21.15.46:0/2707832932 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f896811e060 con 0x7f897806c8b0 2023-12-16T21:43:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.531+0000 7f896f7fe640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8958066470 0x7f8958068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.531+0000 7f896f7fe640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8958066470 0x7f8958068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f897813ca90 tx=0x7f8960009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.535+0000 7f89757fa640 1 -- 172.21.15.46:0/2707832932 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f896814f290 con 0x7f897806c8b0 2023-12-16T21:43:05.697 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f89780b78c0 con 0x7f897806c8b0 2023-12-16T21:43:05.698 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f89757fa640 1 -- 172.21.15.46:0/2707832932 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f89680eb4d0 con 0x7f897806c8b0 2023-12-16T21:43:05.698 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:43:05.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8958066470 msgr2=0x7f8958068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8958066470 0x7f8958068930 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f897813ca90 tx=0x7f8960009040 comp rx=0 tx=0).stop 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 msgr2=0x7f8978141a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f8978141a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f89680ae040 tx=0x7f8968019140 comp rx=0 tx=0).stop 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 shutdown_connections 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8958066470 0x7f8958068930 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f89780af300 0x7f89781e01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f89780ae750 0x7f89781dfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 --2- 172.21.15.46:0/2707832932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f897806c8b0 0x7f8978141a20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 >> 172.21.15.46:0/2707832932 conn(0x7f89780b4900 msgr2=0x7f8978066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 shutdown_connections 2023-12-16T21:43:05.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.691+0000 7f897e62c640 1 -- 172.21.15.46:0/2707832932 wait complete. 2023-12-16T21:43:05.716 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:43:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.835+0000 7fae2904f640 1 Processor -- start 2023-12-16T21:43:05.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.835+0000 7fae2904f640 1 -- start start 2023-12-16T21:43:05.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.835+0000 7fae2904f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240af300 0x7fae240b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.835+0000 7fae2904f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fae240ae750 0x7fae240ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.835+0000 7fae2904f640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae240b02e0 con 0x7fae2406c8b0 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.835+0000 7fae2904f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae240aded0 con 0x7fae240ae750 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.835+0000 7fae2904f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2406cc90 con 0x7fae240af300 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2359e640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fae240ae750 0x7fae240ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae22d9d640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fae2406c8b0 0x7fae240ace20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:49142/0 (socket says 172.21.15.46:49142) 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae22d9d640 1 -- 172.21.15.46:0/677644254 learned_addr learned my addr 172.21.15.46:0/677644254 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2259c640 1 --2- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240af300 0x7fae240b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2359e640 1 -- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240af300 msgr2=0x7fae240b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:05.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2359e640 1 --2- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240af300 0x7fae240b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2359e640 1 -- 172.21.15.46:0/677644254 >> v1:172.21.15.161:6789/0 conn(0x7fae2406c8b0 legacy=0x7fae240ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2359e640 1 -- 172.21.15.46:0/677644254 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae240b05c0 con 0x7fae240ae750 2023-12-16T21:43:05.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2359e640 1 --2- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae240ae750 0x7fae240ad530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009d90 tx=0x7fae1c01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea809cd81973a11d server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae21d9b640 1 -- 172.21.15.46:0/677644254 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0a58c0 con 0x7fae240ae750 2023-12-16T21:43:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae21d9b640 1 -- 172.21.15.46:0/677644254 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c0a5a20 con 0x7fae240ae750 2023-12-16T21:43:05.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae21d9b640 1 -- 172.21.15.46:0/677644254 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0a5db0 con 0x7fae240ae750 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2904f640 1 -- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae240ae750 msgr2=0x7fae240ad530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.839+0000 7fae2904f640 1 --2- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae240ae750 0x7fae240ad530 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae1c009d90 tx=0x7fae1c01c800 comp rx=0 tx=0).stop 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- 172.21.15.46:0/677644254 shutdown_connections 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 --2- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae240ae750 0x7fae240ad530 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 --2- 172.21.15.46:0/677644254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240af300 0x7fae240b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- 172.21.15.46:0/677644254 >> 172.21.15.46:0/677644254 conn(0x7fae240b4900 msgr2=0x7fae240b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- 172.21.15.46:0/677644254 shutdown_connections 2023-12-16T21:43:05.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- 172.21.15.46:0/677644254 wait complete. 2023-12-16T21:43:05.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 Processor -- start 2023-12-16T21:43:05.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- start start 2023-12-16T21:43:05.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2406c8b0 0x7fae241de500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240ae750 0x7fae2413fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae240af300 0x7fae2414b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2414ce30 con 0x7fae240af300 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae241def00 con 0x7fae2406c8b0 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae241df1e0 con 0x7fae240ae750 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae240af300 0x7fae2414b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae240af300 0x7fae2414b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49654/0 (socket says 172.21.15.46:49654) 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 -- 172.21.15.46:0/484979582 learned_addr learned my addr 172.21.15.46:0/484979582 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2259c640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240ae750 0x7fae2413fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae22d9d640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2406c8b0 0x7fae241de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 -- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240ae750 msgr2=0x7fae2413fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240ae750 0x7fae2413fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 -- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2406c8b0 msgr2=0x7fae241de500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2406c8b0 0x7fae241de500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 -- 172.21.15.46:0/484979582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c009a70 con 0x7fae240af300 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2259c640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240ae750 0x7fae2413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2359e640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae240af300 0x7fae2414b2a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae1c01a430 tx=0x7fae1c0195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae0b7fe640 1 -- 172.21.15.46:0/484979582 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0ad640 con 0x7fae240af300 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae0b7fe640 1 -- 172.21.15.46:0/484979582 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae1c0ad7a0 con 0x7fae240af300 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae0b7fe640 1 -- 172.21.15.46:0/484979582 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1c0adb30 con 0x7fae240af300 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae24099eb0 con 0x7fae240af300 2023-12-16T21:43:05.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.843+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2414a080 con 0x7fae240af300 2023-12-16T21:43:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.847+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae24067f70 con 0x7fae240af300 2023-12-16T21:43:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.847+0000 7fae0b7fe640 1 -- 172.21.15.46:0/484979582 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fae1c0b4070 con 0x7fae240af300 2023-12-16T21:43:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.847+0000 7fae0b7fe640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fae10066470 0x7fae10068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.847+0000 7fae0b7fe640 1 -- 172.21.15.46:0/484979582 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae1c11ec40 con 0x7fae240af300 2023-12-16T21:43:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.847+0000 7fae22d9d640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fae10066470 0x7fae10068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:05.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.847+0000 7fae22d9d640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fae10066470 0x7fae10068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae14005890 tx=0x7fae14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:05.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.851+0000 7fae0b7fe640 1 -- 172.21.15.46:0/484979582 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fae1c0ec150 con 0x7fae240af300 2023-12-16T21:43:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.999+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae241df400 con 0x7fae240af300 2023-12-16T21:43:06.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.999+0000 7fae0b7fe640 1 -- 172.21.15.46:0/484979582 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae1c0b2020 con 0x7fae240af300 2023-12-16T21:43:06.007 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.999+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fae10066470 msgr2=0x7fae10068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.999+0000 7fae2904f640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fae10066470 0x7fae10068930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fae14005890 tx=0x7fae14009040 comp rx=0 tx=0).stop 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.999+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae240af300 msgr2=0x7fae2414b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:05.999+0000 7fae2904f640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae240af300 0x7fae2414b2a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fae1c01a430 tx=0x7fae1c0195a0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 shutdown_connections 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fae10066470 0x7fae10068930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae240af300 0x7fae2414b2a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fae240ae750 0x7fae2413fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 --2- 172.21.15.46:0/484979582 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fae2406c8b0 0x7fae241de500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 >> 172.21.15.46:0/484979582 conn(0x7fae240b4900 msgr2=0x7fae2406edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 shutdown_connections 2023-12-16T21:43:06.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.003+0000 7fae2904f640 1 -- 172.21.15.46:0/484979582 wait complete. 2023-12-16T21:43:06.025 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-16T21:43:06.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 Processor -- start 2023-12-16T21:43:06.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 -- start start 2023-12-16T21:43:06.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f40b60b0 0x7f32f4149c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32f40b4a20 0x7f32f406d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f32f414a420 0x7f32f4146700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f406ba40 con 0x7f32f40b4a20 2023-12-16T21:43:06.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f406cb70 con 0x7f32f414a420 2023-12-16T21:43:06.139 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f89dc640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f406da30 con 0x7f32f40b60b0 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32f40b4a20 0x7f32f406d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f32f40b4a20 0x7f32f406d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49664/0 (socket says 172.21.15.46:49664) 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 -- 172.21.15.46:0/96295399 learned_addr learned my addr 172.21.15.46:0/96295399 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f259c640 1 --2- 172.21.15.46:0/96295399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f40b60b0 0x7f32f4149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 -- 172.21.15.46:0/96295399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f40b60b0 msgr2=0x7f32f4149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 --2- 172.21.15.46:0/96295399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f40b60b0 0x7f32f4149c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 -- 172.21.15.46:0/96295399 >> v2:172.21.15.161:3300/0 conn(0x7f32f414a420 msgr2=0x7f32f4146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f2d9d640 1 --2- 172.21.15.46:0/96295399 >> v2:172.21.15.161:3300/0 conn(0x7f32f414a420 0x7f32f4146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 --2- 172.21.15.46:0/96295399 >> v2:172.21.15.161:3300/0 conn(0x7f32f414a420 0x7f32f4146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f1d9b640 1 -- 172.21.15.46:0/96295399 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f406c840 con 0x7f32f40b4a20 2023-12-16T21:43:06.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.131+0000 7f32f2d9d640 1 --2- 172.21.15.46:0/96295399 >> v2:172.21.15.161:3300/0 conn(0x7f32f414a420 0x7f32f4146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f1d9b640 1 --2- 172.21.15.46:0/96295399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f406d000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32f406d850 tx=0x7f32e401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbac654ab7d40e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f159a640 1 -- 172.21.15.46:0/96295399 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40ab600 con 0x7f32f40b4a20 2023-12-16T21:43:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f159a640 1 -- 172.21.15.46:0/96295399 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e40a7050 con 0x7f32f40b4a20 2023-12-16T21:43:06.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f159a640 1 -- 172.21.15.46:0/96295399 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40ab980 con 0x7f32f40b4a20 2023-12-16T21:43:06.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- 172.21.15.46:0/96295399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 msgr2=0x7f32f406d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 --2- 172.21.15.46:0/96295399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f406d000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f32f406d850 tx=0x7f32e401c470 comp rx=0 tx=0).stop 2023-12-16T21:43:06.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- 172.21.15.46:0/96295399 shutdown_connections 2023-12-16T21:43:06.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 --2- 172.21.15.46:0/96295399 >> v2:172.21.15.161:3300/0 conn(0x7f32f414a420 0x7f32f4146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 --2- 172.21.15.46:0/96295399 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f406d000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 --2- 172.21.15.46:0/96295399 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f40b60b0 0x7f32f4149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- 172.21.15.46:0/96295399 >> 172.21.15.46:0/96295399 conn(0x7f32f40ba020 msgr2=0x7f32f40ba430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- 172.21.15.46:0/96295399 shutdown_connections 2023-12-16T21:43:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- 172.21.15.46:0/96295399 wait complete. 2023-12-16T21:43:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 Processor -- start 2023-12-16T21:43:06.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- start start 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f41464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32f40b60b0 0x7f32f41392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f414a420 0x7f32f4144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4144570 con 0x7f32f40b4a20 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f4141210 con 0x7f32f40b60b0 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f89dc640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f41e2a30 con 0x7f32f414a420 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f41464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f41464c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49678/0 (socket says 172.21.15.46:49678) 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 -- 172.21.15.46:0/974232485 learned_addr learned my addr 172.21.15.46:0/974232485 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f2d9d640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f414a420 0x7f32f4144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f1d9b640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32f40b60b0 0x7f32f41392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 -- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f414a420 msgr2=0x7f32f4144f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f414a420 0x7f32f4144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 -- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32f40b60b0 msgr2=0x7f32f41392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32f40b60b0 0x7f32f41392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.135+0000 7f32f259c640 1 -- 172.21.15.46:0/974232485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40094d0 con 0x7f32f40b4a20 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32f259c640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f41464c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c7f0 tx=0x7f32e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32e2ffd640 1 -- 172.21.15.46:0/974232485 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8004220 con 0x7f32f40b4a20 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32e2ffd640 1 -- 172.21.15.46:0/974232485 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e8004380 con 0x7f32f40b4a20 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32e2ffd640 1 -- 172.21.15.46:0/974232485 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8003860 con 0x7f32f40b4a20 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f4142150 con 0x7f32f40b4a20 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f41e2740 con 0x7f32f40b4a20 2023-12-16T21:43:06.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f4141070 con 0x7f32f40b4a20 2023-12-16T21:43:06.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32e2ffd640 1 -- 172.21.15.46:0/974232485 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f32e8003bd0 con 0x7f32f40b4a20 2023-12-16T21:43:06.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32e2ffd640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f32d4066470 0x7f32d4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32e2ffd640 1 -- 172.21.15.46:0/974232485 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32e8083d80 con 0x7f32f40b4a20 2023-12-16T21:43:06.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32f1d9b640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f32d4066470 0x7f32d4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.139+0000 7f32f1d9b640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f32d4066470 0x7f32d4068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f32f406ee30 tx=0x7f32e40a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.143+0000 7f32e2ffd640 1 -- 172.21.15.46:0/974232485 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f32f4141070 con 0x7f32f40b4a20 2023-12-16T21:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.311+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f32f4141070 con 0x7f32f40b4a20 2023-12-16T21:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.311+0000 7f32e2ffd640 1 -- 172.21.15.46:0/974232485 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f32f4141070 con 0x7f32f40b4a20 2023-12-16T21:43:06.319 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f32d4066470 msgr2=0x7f32d4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f32d4066470 0x7f32d4068930 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f32f406ee30 tx=0x7f32e40a6000 comp rx=0 tx=0).stop 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 msgr2=0x7f32f41464c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f41464c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c7f0 tx=0x7f32e800ccb0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 shutdown_connections 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f32d4066470 0x7f32d4068930 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f32f414a420 0x7f32f4144f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f32f40b60b0 0x7f32f41392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 --2- 172.21.15.46:0/974232485 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f32f40b4a20 0x7f32f41464c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 >> 172.21.15.46:0/974232485 conn(0x7f32f40ba020 msgr2=0x7f32f4149550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 shutdown_connections 2023-12-16T21:43:06.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.315+0000 7f32f89dc640 1 -- 172.21.15.46:0/974232485 wait complete. 2023-12-16T21:43:06.338 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-16T21:43:06.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:06.339 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:06.362 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:43:06.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:43:06.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:06.362 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-16T21:43:06.415 INFO:tasks.cephfs.cephfs_test_case:[] 2023-12-16T21:43:06.415 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.571+0000 7f29a94f4640 1 Processor -- start 2023-12-16T21:43:06.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.571+0000 7f29a94f4640 1 -- start start 2023-12-16T21:43:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a94f4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 0x7f29a40ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a94f4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a40b27e0 con 0x7f29a40b3230 2023-12-16T21:43:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a94f4640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a40b2300 con 0x7f29a40b17a0 2023-12-16T21:43:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a94f4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40afe40 con 0x7f29a40ae5b0 2023-12-16T21:43:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f299bfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f29a40b3230 0x7f29a40b3670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55244/0 (socket says 172.21.15.46:55244) 2023-12-16T21:43:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f299bfff640 1 -- 172.21.15.46:0/2285720439 learned_addr learned my addr 172.21.15.46:0/2285720439 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937211497 0 0) 0x7f29a40b27e0 con 0x7f29a40b3230 2023-12-16T21:43:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2990003530 con 0x7f29a40b3230 2023-12-16T21:43:06.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 0x7f29a40ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 806003901 0 0) 0x7f2990003530 con 0x7f29a40b3230 2023-12-16T21:43:06.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 msgr2=0x7f29a40ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:06.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 0x7f29a40ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 >> v1:172.21.15.161:6789/0 conn(0x7f29a40b17a0 legacy=0x7f29a40b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a40b2a30 con 0x7f29a40b3230 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f298c001680 con 0x7f29a40b3230 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 >> v1:172.21.15.46:6789/0 conn(0x7f29a40b3230 legacy=0x7f29a40b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29900039b0 0x7f2990003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 0x7f29900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2990004d10 0x7f2990005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40afe40 con 0x7f29900039b0 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2990005b40 con 0x7f2990004d10 2023-12-16T21:43:06.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2990005d00 con 0x7f29a40ae5b0 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29900039b0 0x7f2990003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f299bfff640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 0x7f29900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a2ffd640 1 -- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 msgr2=0x7f29900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29a40ae5b0 0x7f29900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a2ffd640 1 -- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2990004d10 msgr2=0x7f2990005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a37fe640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2990004d10 0x7f2990005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2990004d10 0x7f2990005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a2ffd640 1 -- 172.21.15.46:0/2285720439 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2990003530 con 0x7f29900039b0 2023-12-16T21:43:06.586 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.575+0000 7f29a37fe640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2990004d10 0x7f2990005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:06.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29900039b0 0x7f2990003dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f299400ba30 tx=0x7f299401c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e884bb6e224b3870 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940ae670 con 0x7f29900039b0 2023-12-16T21:43:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29940ae7d0 con 0x7f29900039b0 2023-12-16T21:43:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a27fc640 1 -- 172.21.15.46:0/2285720439 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29940aeb80 con 0x7f29900039b0 2023-12-16T21:43:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- 172.21.15.46:0/2285720439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29900039b0 msgr2=0x7f2990003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 --2- 172.21.15.46:0/2285720439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29900039b0 0x7f2990003dd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f299400ba30 tx=0x7f299401c650 comp rx=0 tx=0).stop 2023-12-16T21:43:06.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a37fe640 1 -- 172.21.15.46:0/2285720439 reap_dead start 2023-12-16T21:43:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- 172.21.15.46:0/2285720439 shutdown_connections 2023-12-16T21:43:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- 172.21.15.46:0/2285720439 >> 172.21.15.46:0/2285720439 conn(0x7f29a40b48e0 msgr2=0x7f29a40b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- 172.21.15.46:0/2285720439 shutdown_connections 2023-12-16T21:43:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- 172.21.15.46:0/2285720439 wait complete. 2023-12-16T21:43:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 Processor -- start 2023-12-16T21:43:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- start start 2023-12-16T21:43:06.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29900039b0 0x7f29a41dec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a409a420 0x7f29a409a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a414b820 0x7f29a414bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4099b90 con 0x7f29a409a420 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a4099de0 con 0x7f29a414b820 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a409b980 con 0x7f29900039b0 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a409a420 0x7f29a409a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a409a420 0x7f29a409a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49698/0 (socket says 172.21.15.46:49698) 2023-12-16T21:43:06.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 -- 172.21.15.46:0/114224650 learned_addr learned my addr 172.21.15.46:0/114224650 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a37fe640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a414b820 0x7f29a414bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29900039b0 0x7f29a41dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 -- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29900039b0 msgr2=0x7f29a41dec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29900039b0 0x7f29a41dec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 -- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a414b820 msgr2=0x7f29a414bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a414b820 0x7f29a414bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 -- 172.21.15.46:0/114224650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a414ca70 con 0x7f29a409a420 2023-12-16T21:43:06.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a37fe640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a414b820 0x7f29a414bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29900039b0 0x7f29a41dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299bfff640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a409a420 0x7f29a409a8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f298c00e850 tx=0x7f298c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299b7fe640 1 -- 172.21.15.46:0/114224650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00c8d0 con 0x7f29a409a420 2023-12-16T21:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299b7fe640 1 -- 172.21.15.46:0/114224650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c00ca50 con 0x7f29a409a420 2023-12-16T21:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a414cd50 con 0x7f29a409a420 2023-12-16T21:43:06.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f299b7fe640 1 -- 172.21.15.46:0/114224650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00ce00 con 0x7f29a409a420 2023-12-16T21:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.579+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a414c1e0 con 0x7f29a409a420 2023-12-16T21:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.583+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a40679e0 con 0x7f29a409a420 2023-12-16T21:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.583+0000 7f299b7fe640 1 -- 172.21.15.46:0/114224650 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f298c01d460 con 0x7f29a409a420 2023-12-16T21:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.583+0000 7f299b7fe640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2984066470 0x7f2984068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.583+0000 7f299b7fe640 1 -- 172.21.15.46:0/114224650 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f298c083eb0 con 0x7f29a409a420 2023-12-16T21:43:06.596 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.583+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2984066470 0x7f2984068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.583+0000 7f29a2ffd640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2984066470 0x7f2984068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29940b0000 tx=0x7f29940046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.597 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.587+0000 7f299b7fe640 1 -- 172.21.15.46:0/114224650 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f298c051320 con 0x7f29a409a420 2023-12-16T21:43:06.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:43:06.621 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:43:06.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.767+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29a40688f0 con 0x7f29a409a420 2023-12-16T21:43:06.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.767+0000 7f299b7fe640 1 -- 172.21.15.46:0/114224650 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7f298c054940 con 0x7f29a409a420 2023-12-16T21:43:06.776 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:06.777 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:43:06.777 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:43:06.778 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2984066470 msgr2=0x7f2984068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2984066470 0x7f2984068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29940b0000 tx=0x7f29940046d0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a409a420 msgr2=0x7f29a409a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a409a420 0x7f29a409a8a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f298c00e850 tx=0x7f298c00ed10 comp rx=0 tx=0).stop 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 shutdown_connections 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2984066470 0x7f2984068930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f29a414b820 0x7f29a414bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29a409a420 0x7f29a409a8a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 --2- 172.21.15.46:0/114224650 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f29900039b0 0x7f29a41dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 >> 172.21.15.46:0/114224650 conn(0x7f29a40b48e0 msgr2=0x7f29a40b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 shutdown_connections 2023-12-16T21:43:06.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.771+0000 7f29a94f4640 1 -- 172.21.15.46:0/114224650 wait complete. 2023-12-16T21:43:06.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:06.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.895+0000 7fa099d0a640 1 Processor -- start 2023-12-16T21:43:06.905 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.895+0000 7fa099d0a640 1 -- start start 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa099d0a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0940af2e0 0x7fa094147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa099d0a640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa0940ad7c0 0x7fa0941475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa099d0a640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0940af770 con 0x7fa09406c920 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa099d0a640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940afc20 con 0x7fa0940ad7c0 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa099d0a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940aeea0 con 0x7fa0940af2e0 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa093fff640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa0940ad7c0 0x7fa0941475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0940af2e0 0x7fa094147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa0937fe640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7fa09406c920 0x7fa0940ad0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:49174/0 (socket says 172.21.15.46:49174) 2023-12-16T21:43:06.907 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa0937fe640 1 -- 172.21.15.46:0/408944611 learned_addr learned my addr 172.21.15.46:0/408944611 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa093fff640 1 -- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0940af2e0 msgr2=0x7fa094147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa093fff640 1 --2- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0940af2e0 0x7fa094147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa093fff640 1 -- 172.21.15.46:0/408944611 >> v1:172.21.15.161:6789/0 conn(0x7fa09406c920 legacy=0x7fa0940ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa093fff640 1 -- 172.21.15.46:0/408944611 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa094147b00 con 0x7fa0940ad7c0 2023-12-16T21:43:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.899+0000 7fa092ffd640 1 --2- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0940af2e0 0x7fa094147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:06.908 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa093fff640 1 --2- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0940ad7c0 0x7fa0941475c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0097c0 tx=0x7fa08c01c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fd53b20b0e9a88 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa0927fc640 1 -- 172.21.15.46:0/408944611 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0ad660 con 0x7fa0940ad7c0 2023-12-16T21:43:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa0927fc640 1 -- 172.21.15.46:0/408944611 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c0ad7c0 con 0x7fa0940ad7c0 2023-12-16T21:43:06.909 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa0927fc640 1 -- 172.21.15.46:0/408944611 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0adb50 con 0x7fa0940ad7c0 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0940ad7c0 msgr2=0x7fa0941475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 --2- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0940ad7c0 0x7fa0941475c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0097c0 tx=0x7fa08c01c360 comp rx=0 tx=0).stop 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- 172.21.15.46:0/408944611 shutdown_connections 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 --2- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa0940ad7c0 0x7fa0941475c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 --2- 172.21.15.46:0/408944611 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa0940af2e0 0x7fa094147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- 172.21.15.46:0/408944611 >> 172.21.15.46:0/408944611 conn(0x7fa0940b48e0 msgr2=0x7fa0940b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- 172.21.15.46:0/408944611 shutdown_connections 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- 172.21.15.46:0/408944611 wait complete. 2023-12-16T21:43:06.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 Processor -- start 2023-12-16T21:43:06.911 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- start start 2023-12-16T21:43:06.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa09406c920 0x7fa0940b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0940ad7c0 0x7fa094143a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa094143f90 0x7fa094139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09409daa0 con 0x7fa0940ad7c0 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09409d850 con 0x7fa094143f90 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0941466c0 con 0x7fa09406c920 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0940ad7c0 0x7fa094143a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0940ad7c0 0x7fa094143a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49712/0 (socket says 172.21.15.46:49712) 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 -- 172.21.15.46:0/2096165284 learned_addr learned my addr 172.21.15.46:0/2096165284 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa0937fe640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa09406c920 0x7fa0940b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa093fff640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa094143f90 0x7fa094139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 -- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa09406c920 msgr2=0x7fa0940b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa09406c920 0x7fa0940b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 -- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa094143f90 msgr2=0x7fa094139e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:06.913 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa094143f90 0x7fa094139e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 -- 172.21.15.46:0/2096165284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa084009420 con 0x7fa0940ad7c0 2023-12-16T21:43:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa092ffd640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0940ad7c0 0x7fa094143a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa08400e9b0 tx=0x7fa08400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa08bfff640 1 -- 172.21.15.46:0/2096165284 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08400ca00 con 0x7fa0940ad7c0 2023-12-16T21:43:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa08bfff640 1 -- 172.21.15.46:0/2096165284 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08400cb60 con 0x7fa0940ad7c0 2023-12-16T21:43:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08c009420 con 0x7fa0940ad7c0 2023-12-16T21:43:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa08bfff640 1 -- 172.21.15.46:0/2096165284 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084013470 con 0x7fa0940ad7c0 2023-12-16T21:43:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.903+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09409a690 con 0x7fa0940ad7c0 2023-12-16T21:43:06.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.907+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0940679e0 con 0x7fa0940ad7c0 2023-12-16T21:43:06.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.907+0000 7fa08bfff640 1 -- 172.21.15.46:0/2096165284 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fa0840135d0 con 0x7fa0940ad7c0 2023-12-16T21:43:06.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.907+0000 7fa08bfff640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa074066470 0x7fa074068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:06.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.907+0000 7fa08bfff640 1 -- 172.21.15.46:0/2096165284 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0840844e0 con 0x7fa0940ad7c0 2023-12-16T21:43:06.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.907+0000 7fa0937fe640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa074066470 0x7fa074068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:06.915 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.907+0000 7fa0937fe640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa074066470 0x7fa074068930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa09413c220 tx=0x7fa07c001a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:06.911+0000 7fa08bfff640 1 -- 172.21.15.46:0/2096165284 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fa084051970 con 0x7fa0940ad7c0 2023-12-16T21:43:07.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.059+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa094146800 con 0x7fa0940ad7c0 2023-12-16T21:43:07.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.063+0000 7fa08bfff640 1 -- 172.21.15.46:0/2096165284 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fa094146800 con 0x7fa0940ad7c0 2023-12-16T21:43:07.071 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:07.071 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa074066470 msgr2=0x7fa074068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa074066470 0x7fa074068930 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa09413c220 tx=0x7fa07c001a40 comp rx=0 tx=0).stop 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0940ad7c0 msgr2=0x7fa094143a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0940ad7c0 0x7fa094143a50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa08400e9b0 tx=0x7fa08400ee70 comp rx=0 tx=0).stop 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 shutdown_connections 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa074066470 0x7fa074068930 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa094143f90 0x7fa094139e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa0940ad7c0 0x7fa094143a50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 --2- 172.21.15.46:0/2096165284 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa09406c920 0x7fa0940b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 >> 172.21.15.46:0/2096165284 conn(0x7fa0940b48e0 msgr2=0x7fa0940b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:07.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 shutdown_connections 2023-12-16T21:43:07.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.067+0000 7fa099d0a640 1 -- 172.21.15.46:0/2096165284 wait complete. 2023-12-16T21:43:07.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:42:38.743410+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:42:39.749094+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:43:07.089 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:43:07.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.195+0000 7f82b8f1e640 1 Processor -- start 2023-12-16T21:43:07.204 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.195+0000 7f82b8f1e640 1 -- start start 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b8f1e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 0x7f82b40afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b8f1e640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f82b4070a30 0x7f82b4146910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b8f1e640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b406f7c0 con 0x7f82b40b29f0 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b8f1e640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b40711a0 con 0x7f82b4070a30 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b8f1e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40b6210 con 0x7f82b40b5e30 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f82b40b29f0 0x7f82b40b3750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55246/0 (socket says 172.21.15.46:55246) 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 -- 172.21.15.46:0/8409624 learned_addr learned my addr 172.21.15.46:0/8409624 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b259c640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 0x7f82b40afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947431926 0 0) 0x7f82b406f7c0 con 0x7f82b40b29f0 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b1d9b640 1 --2- 172.21.15.46:0/8409624 >> v2:172.21.15.161:3300/0 conn(0x7f82b4070a30 0x7f82b4146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f829c003530 con 0x7f82b40b29f0 2023-12-16T21:43:07.207 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1182219901 0 0) 0x7f829c003530 con 0x7f82b40b29f0 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 msgr2=0x7f82b40afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 0x7f82b40afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 >> v2:172.21.15.161:3300/0 conn(0x7f82b4070a30 msgr2=0x7f82b4146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 --2- 172.21.15.46:0/8409624 >> v2:172.21.15.161:3300/0 conn(0x7f82b4070a30 0x7f82b4146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4070ec0 con 0x7f82b40b29f0 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f82ac001a00 con 0x7f82b40b29f0 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 >> v1:172.21.15.46:6789/0 conn(0x7f82b40b29f0 legacy=0x7f82b40b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 0x7f829c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f829c003f00 0x7f829c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c004d10 0x7f829c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40711a0 con 0x7f829c004d10 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40b6210 con 0x7f829c003f00 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829c00ba70 con 0x7f82b40b5e30 2023-12-16T21:43:07.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c004d10 0x7f829c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b1d9b640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f829c003f00 0x7f829c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b259c640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 0x7f829c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 -- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 msgr2=0x7f829c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82b40b5e30 0x7f829c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 -- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f829c003f00 msgr2=0x7f829c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f829c003f00 0x7f829c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.209 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.199+0000 7f82b2d9d640 1 -- 172.21.15.46:0/8409624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f829c003530 con 0x7f829c004d10 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c004d10 0x7f829c005190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00a860 tx=0x7f82ac01d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f35cc99353fc8ae7 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0ad430 con 0x7f829c004d10 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac0a8070 con 0x7f829c004d10 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b159a640 1 -- 172.21.15.46:0/8409624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0ad850 con 0x7f829c004d10 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- 172.21.15.46:0/8409624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c004d10 msgr2=0x7f829c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 --2- 172.21.15.46:0/8409624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f829c004d10 0x7f829c005190 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00a860 tx=0x7f82ac01d330 comp rx=0 tx=0).stop 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 -- 172.21.15.46:0/8409624 reap_dead start 2023-12-16T21:43:07.210 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- 172.21.15.46:0/8409624 shutdown_connections 2023-12-16T21:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- 172.21.15.46:0/8409624 >> 172.21.15.46:0/8409624 conn(0x7f82b406b860 msgr2=0x7f82b406bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- 172.21.15.46:0/8409624 shutdown_connections 2023-12-16T21:43:07.211 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- 172.21.15.46:0/8409624 wait complete. 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 Processor -- start 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- start start 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f829c004d10 0x7f82b40b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b4144500 0x7f82b41e27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b40b1dd0 0x7f82b40aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b40b1760 con 0x7f82b40b1dd0 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b4137c30 con 0x7f82b4144500 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b4145640 con 0x7f829c004d10 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b40b1dd0 0x7f82b40aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.213 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b40b1dd0 0x7f82b40aebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49738/0 (socket says 172.21.15.46:49738) 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 -- 172.21.15.46:0/2901787162 learned_addr learned my addr 172.21.15.46:0/2901787162 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b259c640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f829c004d10 0x7f82b40b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b1d9b640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b4144500 0x7f82b41e27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 -- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f829c004d10 msgr2=0x7f82b40b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f829c004d10 0x7f82b40b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 -- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b4144500 msgr2=0x7f82b41e27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b4144500 0x7f82b41e27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 -- 172.21.15.46:0/2901787162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4145890 con 0x7f82b40b1dd0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b1d9b640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b4144500 0x7f82b41e27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b2d9d640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b40b1dd0 0x7f82b40aebd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f82ac01cda0 tx=0x7f829c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82a2ffd640 1 -- 172.21.15.46:0/2901787162 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0a5e00 con 0x7f82b40b1dd0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82a2ffd640 1 -- 172.21.15.46:0/2901787162 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac0017c0 con 0x7f82b40b1dd0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b4145040 con 0x7f82b40b1dd0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82a2ffd640 1 -- 172.21.15.46:0/2901787162 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0b6480 con 0x7f82b40b1dd0 2023-12-16T21:43:07.214 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.203+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b4139ef0 con 0x7f82b40b1dd0 2023-12-16T21:43:07.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.207+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b4068960 con 0x7f82b40b1dd0 2023-12-16T21:43:07.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.207+0000 7f82a2ffd640 1 -- 172.21.15.46:0/2901787162 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f82ac0aa030 con 0x7f82b40b1dd0 2023-12-16T21:43:07.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.207+0000 7f82a2ffd640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8294066470 0x7f8294068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.207+0000 7f82a2ffd640 1 -- 172.21.15.46:0/2901787162 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82ac11dc10 con 0x7f82b40b1dd0 2023-12-16T21:43:07.215 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.207+0000 7f82b259c640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8294066470 0x7f8294068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.216 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.207+0000 7f82b259c640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8294066470 0x7f8294068930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009a30 tx=0x7f82a4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.218 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.211+0000 7f82a2ffd640 1 -- 172.21.15.46:0/2901787162 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f82ac0eb090 con 0x7f82b40b1dd0 2023-12-16T21:43:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.359+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82b4141b70 con 0x7f82b40b1dd0 2023-12-16T21:43:07.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.359+0000 7f82a2ffd640 1 -- 172.21.15.46:0/2901787162 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82ac0ee6b0 con 0x7f82b40b1dd0 2023-12-16T21:43:07.367 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8294066470 msgr2=0x7f8294068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8294066470 0x7f8294068930 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f82a4009a30 tx=0x7f82a4008040 comp rx=0 tx=0).stop 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b40b1dd0 msgr2=0x7f82b40aebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b40b1dd0 0x7f82b40aebd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f82ac01cda0 tx=0x7f829c005800 comp rx=0 tx=0).stop 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 shutdown_connections 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8294066470 0x7f8294068930 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82b40b1dd0 0x7f82b40aebd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82b4144500 0x7f82b41e27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 --2- 172.21.15.46:0/2901787162 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f829c004d10 0x7f82b40b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 >> 172.21.15.46:0/2901787162 conn(0x7f82b406b860 msgr2=0x7f82b4143c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 shutdown_connections 2023-12-16T21:43:07.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.363+0000 7f82b8f1e640 1 -- 172.21.15.46:0/2901787162 wait complete. 2023-12-16T21:43:07.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:43:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958fe70640 1 Processor -- start 2023-12-16T21:43:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958fe70640 1 -- start start 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958fe70640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95880b5d80 0x7f95881403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958fe70640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f958806fc70 0x7f9588071780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958fe70640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95880b6160 con 0x7f95880b4070 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958fe70640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9588071100 con 0x7f958806fc70 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958fe70640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880b4590 con 0x7f95880b5d80 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958dc0c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95880b5d80 0x7f95881403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958d40b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f958806fc70 0x7f9588071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958e40d640 1 --1- >> v1:172.21.15.161:6789/0 conn(0x7f95880b4070 0x7f958806a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.161:6789/0 says I am v1:172.21.15.46:49190/0 (socket says 172.21.15.46:49190) 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958dc0c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95880b5d80 0x7f95881403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:34262/0 (socket says 172.21.15.46:34262) 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958e40d640 1 -- 172.21.15.46:0/298954715 learned_addr learned my addr 172.21.15.46:0/298954715 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958d40b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f958806fc70 0x7f9588071780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:58162/0 (socket says 172.21.15.46:58162) 2023-12-16T21:43:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958dc0c640 1 -- 172.21.15.46:0/298954715 >> v2:172.21.15.161:3300/0 conn(0x7f958806fc70 msgr2=0x7f9588071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958dc0c640 1 --2- 172.21.15.46:0/298954715 >> v2:172.21.15.161:3300/0 conn(0x7f958806fc70 0x7f9588071780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958dc0c640 1 -- 172.21.15.46:0/298954715 >> v1:172.21.15.161:6789/0 conn(0x7f95880b4070 legacy=0x7f958806a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958dc0c640 1 -- 172.21.15.46:0/298954715 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95880b4870 con 0x7f95880b5d80 2023-12-16T21:43:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.503+0000 7f958d40b640 1 --2- 172.21.15.46:0/298954715 >> v2:172.21.15.161:3300/0 conn(0x7f958806fc70 0x7f9588071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958dc0c640 1 --2- 172.21.15.46:0/298954715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95880b5d80 0x7f95881403c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95840097c0 tx=0x7f958401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7909fd9619a8d3c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958cc0a640 1 -- 172.21.15.46:0/298954715 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840ab600 con 0x7f95880b5d80 2023-12-16T21:43:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958cc0a640 1 -- 172.21.15.46:0/298954715 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95840a7050 con 0x7f95880b5d80 2023-12-16T21:43:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958cc0a640 1 -- 172.21.15.46:0/298954715 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95840ab980 con 0x7f95880b5d80 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- 172.21.15.46:0/298954715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95880b5d80 msgr2=0x7f95881403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 --2- 172.21.15.46:0/298954715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95880b5d80 0x7f95881403c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f95840097c0 tx=0x7f958401c3c0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- 172.21.15.46:0/298954715 shutdown_connections 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 --2- 172.21.15.46:0/298954715 >> v2:172.21.15.161:3300/0 conn(0x7f958806fc70 0x7f9588071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 --2- 172.21.15.46:0/298954715 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f95880b5d80 0x7f95881403c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- 172.21.15.46:0/298954715 >> 172.21.15.46:0/298954715 conn(0x7f958806b860 msgr2=0x7f958806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- 172.21.15.46:0/298954715 shutdown_connections 2023-12-16T21:43:07.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- 172.21.15.46:0/298954715 wait complete. 2023-12-16T21:43:07.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 Processor -- start 2023-12-16T21:43:07.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- start start 2023-12-16T21:43:07.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f958806fc70 0x7f95880ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95880b4070 0x7f95881e2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95880b5d80 0x7f95880ad160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95881e3020 con 0x7f95880b5d80 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958813c930 con 0x7f95880b4070 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.507+0000 7f958fe70640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95880adda0 con 0x7f958806fc70 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95880b5d80 0x7f95880ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95880b5d80 0x7f95880ad160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49750/0 (socket says 172.21.15.46:49750) 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 -- 172.21.15.46:0/414992866 learned_addr learned my addr 172.21.15.46:0/414992866 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958d40b640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95880b4070 0x7f95881e2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958dc0c640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f958806fc70 0x7f95880ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 -- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f958806fc70 msgr2=0x7f95880ac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f958806fc70 0x7f95880ac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 -- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95880b4070 msgr2=0x7f95881e2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95880b4070 0x7f95881e2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 -- 172.21.15.46:0/414992866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009420 con 0x7f95880b5d80 2023-12-16T21:43:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958e40d640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95880b5d80 0x7f95880ad160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f958000caf0 tx=0x7f958000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f957e7fc640 1 -- 172.21.15.46:0/414992866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958000aca0 con 0x7f95880b5d80 2023-12-16T21:43:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f957e7fc640 1 -- 172.21.15.46:0/414992866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958000ae00 con 0x7f95880b5d80 2023-12-16T21:43:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f957e7fc640 1 -- 172.21.15.46:0/414992866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9580012600 con 0x7f95880b5d80 2023-12-16T21:43:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9584009420 con 0x7f95880b5d80 2023-12-16T21:43:07.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95880ae2d0 con 0x7f95880b5d80 2023-12-16T21:43:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95881e3170 con 0x7f95880b5d80 2023-12-16T21:43:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f957e7fc640 1 -- 172.21.15.46:0/414992866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f9580007070 con 0x7f95880b5d80 2023-12-16T21:43:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f957e7fc640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f956c066470 0x7f956c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f957e7fc640 1 -- 172.21.15.46:0/414992866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9580083c10 con 0x7f95880b5d80 2023-12-16T21:43:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.511+0000 7f958dc0c640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f956c066470 0x7f956c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.515+0000 7f958dc0c640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f956c066470 0x7f956c068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95840ad000 tx=0x7f95840a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.524 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.515+0000 7f957e7fc640 1 -- 172.21.15.46:0/414992866 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f95881e3170 con 0x7f95880b5d80 2023-12-16T21:43:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.675+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95881e3170 con 0x7f95880b5d80 2023-12-16T21:43:07.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.675+0000 7f957e7fc640 1 -- 172.21.15.46:0/414992866 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95881e3170 con 0x7f95880b5d80 2023-12-16T21:43:07.684 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:43:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f956c066470 msgr2=0x7f956c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f956c066470 0x7f956c068930 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f95840ad000 tx=0x7f95840a6000 comp rx=0 tx=0).stop 2023-12-16T21:43:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95880b5d80 msgr2=0x7f95880ad160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95880b5d80 0x7f95880ad160 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f958000caf0 tx=0x7f958000a060 comp rx=0 tx=0).stop 2023-12-16T21:43:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 shutdown_connections 2023-12-16T21:43:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f956c066470 0x7f956c068930 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95880b5d80 0x7f95880ad160 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f95880b4070 0x7f95881e2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 --2- 172.21.15.46:0/414992866 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f958806fc70 0x7f95880ac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 >> 172.21.15.46:0/414992866 conn(0x7f958806b860 msgr2=0x7f958813f670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 shutdown_connections 2023-12-16T21:43:07.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.679+0000 7f958fe70640 1 -- 172.21.15.46:0/414992866 wait complete. 2023-12-16T21:43:07.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:43:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf7d59d640 1 Processor -- start 2023-12-16T21:43:07.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf7d59d640 1 -- start start 2023-12-16T21:43:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf7d59d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 0x7fdf780b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf7d59d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf780b02e0 con 0x7fdf780ae750 2023-12-16T21:43:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf7d59d640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf780adc40 con 0x7fdf780af300 2023-12-16T21:43:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf7d59d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780aded0 con 0x7fdf7806c8b0 2023-12-16T21:43:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdf780ae750 0x7fdf780ad530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55250/0 (socket says 172.21.15.46:55250) 2023-12-16T21:43:07.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 -- 172.21.15.46:0/1329627387 learned_addr learned my addr 172.21.15.46:0/1329627387 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf76ffd640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 0x7fdf780b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2103286038 0 0) 0x7fdf780b02e0 con 0x7fdf780ae750 2023-12-16T21:43:07.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf64003530 con 0x7fdf780ae750 2023-12-16T21:43:07.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951791684 0 0) 0x7fdf64003530 con 0x7fdf780ae750 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 msgr2=0x7fdf780b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 0x7fdf780b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 >> v1:172.21.15.161:6789/0 conn(0x7fdf780af300 legacy=0x7fdf780ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7806cc90 con 0x7fdf780ae750 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fdf70001a00 con 0x7fdf780ae750 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 >> v1:172.21.15.46:6789/0 conn(0x7fdf780ae750 legacy=0x7fdf780ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 0x7fdf640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf64003f30 0x7fdf640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf64005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf780aded0 con 0x7fdf64004d10 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6400bbd0 con 0x7fdf64003f30 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6400dfa0 con 0x7fdf7806c8b0 2023-12-16T21:43:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf64005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf76ffd640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 0x7fdf640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf6ffff640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf64003f30 0x7fdf640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 -- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 msgr2=0x7fdf640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf7806c8b0 0x7fdf640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 -- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf64003f30 msgr2=0x7fdf640047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf64003f30 0x7fdf640047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf777fe640 1 -- 172.21.15.46:0/1329627387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf64003530 con 0x7fdf64004d10 2023-12-16T21:43:07.834 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.823+0000 7fdf6ffff640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf64003f30 0x7fdf640047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf777fe640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf64005190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000a830 tx=0x7fdf7001d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ac7a581c79bfaa server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf700072d0 con 0x7fdf64004d10 2023-12-16T21:43:07.835 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf700a8070 con 0x7fdf64004d10 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf767fc640 1 -- 172.21.15.46:0/1329627387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70007690 con 0x7fdf64004d10 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1329627387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 msgr2=0x7fdf64005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 --2- 172.21.15.46:0/1329627387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf64005190 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000a830 tx=0x7fdf7001d310 comp rx=0 tx=0).stop 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf777fe640 1 -- 172.21.15.46:0/1329627387 reap_dead start 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1329627387 shutdown_connections 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1329627387 >> 172.21.15.46:0/1329627387 conn(0x7fdf780b4900 msgr2=0x7fdf780b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1329627387 shutdown_connections 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1329627387 wait complete. 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 Processor -- start 2023-12-16T21:43:07.836 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- start start 2023-12-16T21:43:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf78142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf78141ce0 0x7fdf781ddaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf78142160 0x7fdf7806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf781408b0 con 0x7fdf64004d10 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf781de390 con 0x7fdf78142160 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf7d59d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf78141570 con 0x7fdf78141ce0 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf76ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf78142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf76ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf78142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49758/0 (socket says 172.21.15.46:49758) 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.827+0000 7fdf76ffd640 1 -- 172.21.15.46:0/1331103151 learned_addr learned my addr 172.21.15.46:0/1331103151 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6ffff640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf78141ce0 0x7fdf781ddaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf777fe640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf78142160 0x7fdf7806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf76ffd640 1 -- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf78141ce0 msgr2=0x7fdf781ddaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf76ffd640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf78141ce0 0x7fdf781ddaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf76ffd640 1 -- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf78142160 msgr2=0x7fdf7806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf76ffd640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf78142160 0x7fdf7806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf76ffd640 1 -- 172.21.15.46:0/1331103151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf781402d0 con 0x7fdf64004d10 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf76ffd640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf78142c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdf68011640 tx=0x7fdf68011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6f7fe640 1 -- 172.21.15.46:0/1331103151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800f8c0 con 0x7fdf64004d10 2023-12-16T21:43:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6f7fe640 1 -- 172.21.15.46:0/1331103151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6800fa20 con 0x7fdf64004d10 2023-12-16T21:43:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6f7fe640 1 -- 172.21.15.46:0/1331103151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6800fd90 con 0x7fdf64004d10 2023-12-16T21:43:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf781405b0 con 0x7fdf64004d10 2023-12-16T21:43:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf781de820 con 0x7fdf64004d10 2023-12-16T21:43:07.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf78140130 con 0x7fdf64004d10 2023-12-16T21:43:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6f7fe640 1 -- 172.21.15.46:0/1331103151 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fdf68015460 con 0x7fdf64004d10 2023-12-16T21:43:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6f7fe640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf58066470 0x7fdf58068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6f7fe640 1 -- 172.21.15.46:0/1331103151 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf680843b0 con 0x7fdf64004d10 2023-12-16T21:43:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6ffff640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf58066470 0x7fdf58068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:07.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.831+0000 7fdf6ffff640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf58066470 0x7fdf58068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf7813da50 tx=0x7fdf60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:07.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.835+0000 7fdf6f7fe640 1 -- 172.21.15.46:0/1331103151 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fdf78140130 con 0x7fdf64004d10 2023-12-16T21:43:07.998 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.991+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf78140130 con 0x7fdf64004d10 2023-12-16T21:43:07.999 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.991+0000 7fdf6f7fe640 1 -- 172.21.15.46:0/1331103151 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf78140130 con 0x7fdf64004d10 2023-12-16T21:43:07.999 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf58066470 msgr2=0x7fdf58068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf58066470 0x7fdf58068930 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf7813da50 tx=0x7fdf60009040 comp rx=0 tx=0).stop 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 msgr2=0x7fdf78142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf78142c00 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdf68011640 tx=0x7fdf68011b00 comp rx=0 tx=0).stop 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 shutdown_connections 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf58066470 0x7fdf58068930 secure :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf7813da50 tx=0x7fdf60009040 comp rx=0 tx=0).stop 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf78142160 0x7fdf7806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf78141ce0 0x7fdf781ddaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 --2- 172.21.15.46:0/1331103151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf64004d10 0x7fdf78142c00 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 >> 172.21.15.46:0/1331103151 conn(0x7fdf780b4900 msgr2=0x7fdf7813f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:08.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 shutdown_connections 2023-12-16T21:43:08.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:07.995+0000 7fdf7d59d640 1 -- 172.21.15.46:0/1331103151 wait complete. 2023-12-16T21:43:08.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:08.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.135+0000 7f3220830640 1 Processor -- start 2023-12-16T21:43:08.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.135+0000 7f3220830640 1 -- start start 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3220830640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f321c0b3fd0 0x7f321c13e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3220830640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c0af2e0 con 0x7f321c0b0340 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3220830640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f321c0b1f50 con 0x7f321c06c920 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3220830640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321c06cd00 con 0x7f321c0b3fd0 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3219d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f321c0b3fd0 0x7f321c13e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f321ad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f321c0b0340 0x7f321c13e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:55264/0 (socket says 172.21.15.46:55264) 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f321ad9d640 1 -- 172.21.15.46:0/466037365 learned_addr learned my addr 172.21.15.46:0/466037365 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3219d9b640 1 -- 172.21.15.46:0/466037365 >> v1:172.21.15.161:6789/0 conn(0x7f321c06c920 legacy=0x7f321c0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3219d9b640 1 -- 172.21.15.46:0/466037365 >> v1:172.21.15.46:6789/0 conn(0x7f321c0b0340 legacy=0x7f321c13e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3219d9b640 1 -- 172.21.15.46:0/466037365 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c0af530 con 0x7f321c0b3fd0 2023-12-16T21:43:08.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f321959a640 1 -- 172.21.15.46:0/466037365 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833597260 0 0) 0x7f321c0af2e0 con 0x7f321c0b0340 2023-12-16T21:43:08.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f3219d9b640 1 --2- 172.21.15.46:0/466037365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0b3fd0 0x7f321c13e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f320401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d57ab86581eceb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f321959a640 1 -- 172.21.15.46:0/466037365 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32040a5740 con 0x7f321c0b3fd0 2023-12-16T21:43:08.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f321959a640 1 -- 172.21.15.46:0/466037365 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32040a58a0 con 0x7f321c0b3fd0 2023-12-16T21:43:08.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.139+0000 7f321959a640 1 -- 172.21.15.46:0/466037365 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32040a5c30 con 0x7f321c0b3fd0 2023-12-16T21:43:08.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/466037365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0b3fd0 msgr2=0x7f321c13e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 --2- 172.21.15.46:0/466037365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0b3fd0 0x7f321c13e320 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3204009d90 tx=0x7f320401c880 comp rx=0 tx=0).stop 2023-12-16T21:43:08.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/466037365 shutdown_connections 2023-12-16T21:43:08.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 --2- 172.21.15.46:0/466037365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c0b3fd0 0x7f321c13e320 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/466037365 >> 172.21.15.46:0/466037365 conn(0x7f321c0b48e0 msgr2=0x7f321c0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:08.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/466037365 shutdown_connections 2023-12-16T21:43:08.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/466037365 wait complete. 2023-12-16T21:43:08.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 Processor -- start 2023-12-16T21:43:08.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- start start 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f321c06c920 0x7f321c141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f321c144260 0x7f321c143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c143680 0x7f321c1dd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c071070 con 0x7f321c143680 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c145470 con 0x7f321c144260 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c144740 con 0x7f321c06c920 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c143680 0x7f321c1dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c143680 0x7f321c1dd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49782/0 (socket says 172.21.15.46:49782) 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 -- 172.21.15.46:0/3641633391 learned_addr learned my addr 172.21.15.46:0/3641633391 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3219d9b640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f321c144260 0x7f321c143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321a59c640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f321c06c920 0x7f321c141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 -- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f321c06c920 msgr2=0x7f321c141730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f321c06c920 0x7f321c141730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 -- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f321c144260 msgr2=0x7f321c143140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f321c144260 0x7f321c143140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 -- 172.21.15.46:0/3641633391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204009a70 con 0x7f321c143680 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3219d9b640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f321c144260 0x7f321c143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321a59c640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f321c06c920 0x7f321c141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f321ad9d640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c143680 0x7f321c1dd400 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f321400c3c0 tx=0x7f321400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3212ffd640 1 -- 172.21.15.46:0/3641633391 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32140074e0 con 0x7f321c143680 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3212ffd640 1 -- 172.21.15.46:0/3641633391 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3214007640 con 0x7f321c143680 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f321c144eb0 con 0x7f321c143680 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3212ffd640 1 -- 172.21.15.46:0/3641633391 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3214004a30 con 0x7f321c143680 2023-12-16T21:43:08.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c1dcf80 con 0x7f321c143680 2023-12-16T21:43:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.143+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c0679e0 con 0x7f321c143680 2023-12-16T21:43:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.147+0000 7f3212ffd640 1 -- 172.21.15.46:0/3641633391 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f3214004b90 con 0x7f321c143680 2023-12-16T21:43:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.147+0000 7f3212ffd640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31fc066470 0x7f31fc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.147+0000 7f3212ffd640 1 -- 172.21.15.46:0/3641633391 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3214083c10 con 0x7f321c143680 2023-12-16T21:43:08.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.147+0000 7f321a59c640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31fc066470 0x7f31fc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.147+0000 7f321a59c640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31fc066470 0x7f31fc068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f320c003e40 tx=0x7f320c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.157 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.151+0000 7f3212ffd640 1 -- 172.21.15.46:0/3641633391 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f3214051120 con 0x7f321c143680 2023-12-16T21:43:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.335+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f321c0688f0 con 0x7f321c143680 2023-12-16T21:43:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.335+0000 7f3212ffd640 1 -- 172.21.15.46:0/3641633391 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7f3214054740 con 0x7f321c143680 2023-12-16T21:43:08.345 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:08.345 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:43:08.345 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31fc066470 msgr2=0x7f31fc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31fc066470 0x7f31fc068930 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f320c003e40 tx=0x7f320c009040 comp rx=0 tx=0).stop 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c143680 msgr2=0x7f321c1dd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c143680 0x7f321c1dd400 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f321400c3c0 tx=0x7f321400c880 comp rx=0 tx=0).stop 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 shutdown_connections 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31fc066470 0x7f31fc068930 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f321c143680 0x7f321c1dd400 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f321c144260 0x7f321c143140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 --2- 172.21.15.46:0/3641633391 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f321c06c920 0x7f321c141730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 >> 172.21.15.46:0/3641633391 conn(0x7f321c0b48e0 msgr2=0x7f321c13d090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 shutdown_connections 2023-12-16T21:43:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.339+0000 7f3220830640 1 -- 172.21.15.46:0/3641633391 wait complete. 2023-12-16T21:43:08.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb5c011640 1 Processor -- start 2023-12-16T21:43:08.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb5c011640 1 -- start start 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb5c011640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcb540ae5b0 0x7fcb540ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb5c011640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fcb540b3230 0x7fcb540b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb5c011640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb540b27e0 con 0x7fcb540b17a0 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb5c011640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540afe40 con 0x7fcb540ae5b0 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb5c011640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb540b2320 con 0x7fcb540b3230 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb59dad640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcb540ae5b0 0x7fcb540ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb59dad640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fcb540ae5b0 0x7fcb540ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49784/0 (socket says 172.21.15.46:49784) 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb59dad640 1 -- 172.21.15.46:0/4103227869 learned_addr learned my addr 172.21.15.46:0/4103227869 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb595ac640 1 --2- 172.21.15.46:0/4103227869 >> v2:172.21.15.161:3300/0 conn(0x7fcb540b3230 0x7fcb540b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb59dad640 1 -- 172.21.15.46:0/4103227869 >> v2:172.21.15.161:3300/0 conn(0x7fcb540b3230 msgr2=0x7fcb540b36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb59dad640 1 --2- 172.21.15.46:0/4103227869 >> v2:172.21.15.161:3300/0 conn(0x7fcb540b3230 0x7fcb540b36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb59dad640 1 -- 172.21.15.46:0/4103227869 >> v1:172.21.15.161:6789/0 conn(0x7fcb540b17a0 legacy=0x7fcb540b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb59dad640 1 -- 172.21.15.46:0/4103227869 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540b3be0 con 0x7fcb540ae5b0 2023-12-16T21:43:08.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.479+0000 7fcb595ac640 1 --2- 172.21.15.46:0/4103227869 >> v2:172.21.15.161:3300/0 conn(0x7fcb540b3230 0x7fcb540b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb59dad640 1 --2- 172.21.15.46:0/4103227869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb540ae990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb5406cf70 tx=0x7fcb5001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae80cced5a1b8557 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb58dab640 1 -- 172.21.15.46:0/4103227869 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500ac620 con 0x7fcb540ae5b0 2023-12-16T21:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb58dab640 1 -- 172.21.15.46:0/4103227869 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb500ac780 con 0x7fcb540ae5b0 2023-12-16T21:43:08.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb58dab640 1 -- 172.21.15.46:0/4103227869 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500acb10 con 0x7fcb540ae5b0 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 -- 172.21.15.46:0/4103227869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 msgr2=0x7fcb540ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 --2- 172.21.15.46:0/4103227869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb540ae990 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fcb5406cf70 tx=0x7fcb5001c440 comp rx=0 tx=0).stop 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 -- 172.21.15.46:0/4103227869 shutdown_connections 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 --2- 172.21.15.46:0/4103227869 >> v2:172.21.15.161:3300/0 conn(0x7fcb540b3230 0x7fcb540b36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 --2- 172.21.15.46:0/4103227869 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb540ae990 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 -- 172.21.15.46:0/4103227869 >> 172.21.15.46:0/4103227869 conn(0x7fcb540b48e0 msgr2=0x7fcb540b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 -- 172.21.15.46:0/4103227869 shutdown_connections 2023-12-16T21:43:08.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.483+0000 7fcb5c011640 1 -- 172.21.15.46:0/4103227869 wait complete. 2023-12-16T21:43:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 Processor -- start 2023-12-16T21:43:08.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 -- start start 2023-12-16T21:43:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb541385d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb540b17a0 0x7fcb541dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb540b3230 0x7fcb541e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54099dc0 con 0x7fcb540ae5b0 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5409a030 con 0x7fcb540b17a0 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb54070790 con 0x7fcb540b3230 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb541385d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb541385d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49788/0 (socket says 172.21.15.46:49788) 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 -- 172.21.15.46:0/657022823 learned_addr learned my addr 172.21.15.46:0/657022823 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5a5ae640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb540b3230 0x7fcb541e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb595ac640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb540b17a0 0x7fcb541dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 -- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb540b3230 msgr2=0x7fcb541e01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb540b3230 0x7fcb541e01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 -- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb540b17a0 msgr2=0x7fcb541dfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb540b17a0 0x7fcb541dfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 -- 172.21.15.46:0/657022823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500094d0 con 0x7fcb540ae5b0 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5a5ae640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb540b3230 0x7fcb541e01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:08.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb59dad640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb541385d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb500af040 tx=0x7fcb500191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb4a7fc640 1 -- 172.21.15.46:0/657022823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500b7470 con 0x7fcb540ae5b0 2023-12-16T21:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb4a7fc640 1 -- 172.21.15.46:0/657022823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb500b75d0 con 0x7fcb540ae5b0 2023-12-16T21:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb4a7fc640 1 -- 172.21.15.46:0/657022823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb500b7980 con 0x7fcb540ae5b0 2023-12-16T21:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb54070a70 con 0x7fcb540ae5b0 2023-12-16T21:43:08.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb54071030 con 0x7fcb540ae5b0 2023-12-16T21:43:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.487+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb540679e0 con 0x7fcb540ae5b0 2023-12-16T21:43:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.491+0000 7fcb4a7fc640 1 -- 172.21.15.46:0/657022823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fcb500b3070 con 0x7fcb540ae5b0 2023-12-16T21:43:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.491+0000 7fcb4a7fc640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb38066470 0x7fcb38068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.491+0000 7fcb4a7fc640 1 -- 172.21.15.46:0/657022823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb5011e450 con 0x7fcb540ae5b0 2023-12-16T21:43:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.491+0000 7fcb595ac640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb38066470 0x7fcb38068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.491+0000 7fcb595ac640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb38066470 0x7fcb38068930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb54138ce0 tx=0x7fcb44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.495+0000 7fcb4a7fc640 1 -- 172.21.15.46:0/657022823 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fcb500eb960 con 0x7fcb540ae5b0 2023-12-16T21:43:08.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.659+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb54071220 con 0x7fcb540ae5b0 2023-12-16T21:43:08.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb4a7fc640 1 -- 172.21.15.46:0/657022823 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7fcb500eef80 con 0x7fcb540ae5b0 2023-12-16T21:43:08.669 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:08.669 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb38066470 msgr2=0x7fcb38068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb38066470 0x7fcb38068930 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcb54138ce0 tx=0x7fcb44008040 comp rx=0 tx=0).stop 2023-12-16T21:43:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 msgr2=0x7fcb541385d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb541385d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcb500af040 tx=0x7fcb500191e0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 shutdown_connections 2023-12-16T21:43:08.671 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fcb38066470 0x7fcb38068930 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fcb540b3230 0x7fcb541e01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fcb540b17a0 0x7fcb541dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 --2- 172.21.15.46:0/657022823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcb540ae5b0 0x7fcb541385d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 >> 172.21.15.46:0/657022823 conn(0x7fcb540b48e0 msgr2=0x7fcb5406b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 shutdown_connections 2023-12-16T21:43:08.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.663+0000 7fcb5c011640 1 -- 172.21.15.46:0/657022823 wait complete. 2023-12-16T21:43:08.689 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-16T21:43:08.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.799+0000 7fa3b7ded640 1 Processor -- start 2023-12-16T21:43:08.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.799+0000 7fa3b7ded640 1 -- start start 2023-12-16T21:43:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b7ded640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00af2e0 0x7fa3b0147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b7ded640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa3b00ad7c0 0x7fa3b01475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b7ded640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b00af770 con 0x7fa3b006c920 2023-12-16T21:43:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b7ded640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b00afc20 con 0x7fa3b00ad7c0 2023-12-16T21:43:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b7ded640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b00aeea0 con 0x7fa3b00af2e0 2023-12-16T21:43:08.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00af2e0 0x7fa3b0147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b638a640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa3b00ad7c0 0x7fa3b01475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00af2e0 0x7fa3b0147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:38794/0 (socket says 172.21.15.46:38794) 2023-12-16T21:43:08.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 -- 172.21.15.46:0/1296934871 learned_addr learned my addr 172.21.15.46:0/1296934871 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:08.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 -- 172.21.15.46:0/1296934871 >> v2:172.21.15.161:3300/0 conn(0x7fa3b00ad7c0 msgr2=0x7fa3b01475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:08.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 --2- 172.21.15.46:0/1296934871 >> v2:172.21.15.161:3300/0 conn(0x7fa3b00ad7c0 0x7fa3b01475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 -- 172.21.15.46:0/1296934871 >> v1:172.21.15.161:6789/0 conn(0x7fa3b006c920 legacy=0x7fa3b00ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 -- 172.21.15.46:0/1296934871 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0147b00 con 0x7fa3b00af2e0 2023-12-16T21:43:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b5388640 1 --2- 172.21.15.46:0/1296934871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00af2e0 0x7fa3b0147080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009870 tx=0x7fa3ac01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6906019bee90ff11 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b4b87640 1 -- 172.21.15.46:0/1296934871 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0ab620 con 0x7fa3b00af2e0 2023-12-16T21:43:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b4b87640 1 -- 172.21.15.46:0/1296934871 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac0ab780 con 0x7fa3b00af2e0 2023-12-16T21:43:08.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.803+0000 7fa3b4b87640 1 -- 172.21.15.46:0/1296934871 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0abb10 con 0x7fa3b00af2e0 2023-12-16T21:43:08.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- 172.21.15.46:0/1296934871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00af2e0 msgr2=0x7fa3b0147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/1296934871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00af2e0 0x7fa3b0147080 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009870 tx=0x7fa3ac01c420 comp rx=0 tx=0).stop 2023-12-16T21:43:08.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- 172.21.15.46:0/1296934871 shutdown_connections 2023-12-16T21:43:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/1296934871 >> v2:172.21.15.161:3300/0 conn(0x7fa3b00ad7c0 0x7fa3b01475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/1296934871 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00af2e0 0x7fa3b0147080 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- 172.21.15.46:0/1296934871 >> 172.21.15.46:0/1296934871 conn(0x7fa3b00b48e0 msgr2=0x7fa3b00b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- 172.21.15.46:0/1296934871 shutdown_connections 2023-12-16T21:43:08.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- 172.21.15.46:0/1296934871 wait complete. 2023-12-16T21:43:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 Processor -- start 2023-12-16T21:43:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- start start 2023-12-16T21:43:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b006c920 0x7fa3b0146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00ad7c0 0x7fa3b01e0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b006ffc0 0x7fa3b0144f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0070450 con 0x7fa3b006ffc0 2023-12-16T21:43:08.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0143100 con 0x7fa3b006c920 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b0146240 con 0x7fa3b00ad7c0 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b006ffc0 0x7fa3b0144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b006ffc0 0x7fa3b0144f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51842/0 (socket says 172.21.15.46:51842) 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 -- 172.21.15.46:0/3867860487 learned_addr learned my addr 172.21.15.46:0/3867860487 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b5b89640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b006c920 0x7fa3b0146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b5388640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00ad7c0 0x7fa3b01e0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 -- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00ad7c0 msgr2=0x7fa3b01e0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00ad7c0 0x7fa3b01e0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 -- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b006c920 msgr2=0x7fa3b0146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b006c920 0x7fa3b0146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 -- 172.21.15.46:0/3867860487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac0094d0 con 0x7fa3b006ffc0 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b5b89640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b006c920 0x7fa3b0146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:08.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b638a640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b006ffc0 0x7fa3b0144f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800efc0 tx=0x7fa3a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3a27fc640 1 -- 172.21.15.46:0/3867860487 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8014480 con 0x7fa3b006ffc0 2023-12-16T21:43:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3a27fc640 1 -- 172.21.15.46:0/3867860487 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a800f040 con 0x7fa3b006ffc0 2023-12-16T21:43:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b01459a0 con 0x7fa3b006ffc0 2023-12-16T21:43:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3a27fc640 1 -- 172.21.15.46:0/3867860487 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8014820 con 0x7fa3b006ffc0 2023-12-16T21:43:08.818 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.807+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b00707d0 con 0x7fa3b006ffc0 2023-12-16T21:43:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.811+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b00679e0 con 0x7fa3b006ffc0 2023-12-16T21:43:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.811+0000 7fa3a27fc640 1 -- 172.21.15.46:0/3867860487 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fa3a8014980 con 0x7fa3b006ffc0 2023-12-16T21:43:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.811+0000 7fa3a27fc640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa3940664a0 0x7fa394068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.811+0000 7fa3a27fc640 1 -- 172.21.15.46:0/3867860487 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3a80849c0 con 0x7fa3b006ffc0 2023-12-16T21:43:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.811+0000 7fa3b5b89640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa3940664a0 0x7fa394068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:08.819 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.811+0000 7fa3b5b89640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa3940664a0 0x7fa394068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000f70 tx=0x7fa3a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:08.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:08.815+0000 7fa3a27fc640 1 -- 172.21.15.46:0/3867860487 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fa3a8051ed0 con 0x7fa3b006ffc0 2023-12-16T21:43:09.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.003+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3b00688f0 con 0x7fa3b006ffc0 2023-12-16T21:43:09.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.003+0000 7fa3a27fc640 1 -- 172.21.15.46:0/3867860487 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7fa3a8018090 con 0x7fa3b006ffc0 2023-12-16T21:43:09.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:09.011 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:43:09.011 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:43:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa3940664a0 msgr2=0x7fa394068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa3940664a0 0x7fa394068960 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000f70 tx=0x7fa3a4009040 comp rx=0 tx=0).stop 2023-12-16T21:43:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b006ffc0 msgr2=0x7fa3b0144f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b006ffc0 0x7fa3b0144f70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa3a800efc0 tx=0x7fa3a800c490 comp rx=0 tx=0).stop 2023-12-16T21:43:09.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 shutdown_connections 2023-12-16T21:43:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa3940664a0 0x7fa394068960 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3b006ffc0 0x7fa3b0144f70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa3b00ad7c0 0x7fa3b01e0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 --2- 172.21.15.46:0/3867860487 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa3b006c920 0x7fa3b0146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 >> 172.21.15.46:0/3867860487 conn(0x7fa3b00b48e0 msgr2=0x7fa3b00b7280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 shutdown_connections 2023-12-16T21:43:09.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.007+0000 7fa3b7ded640 1 -- 172.21.15.46:0/3867860487 wait complete. 2023-12-16T21:43:09.031 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:09.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 Processor -- start 2023-12-16T21:43:09.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 -- start start 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f937406c920 0x7f93740ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f93740af2e0 0x7f93740ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93740b4430 0x7f9374147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93740afc20 con 0x7f93740b4430 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93740af770 con 0x7f93740af2e0 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9378d4e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93740aeea0 con 0x7f937406c920 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93740b4430 0x7f9374147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f93740b4430 0x7f9374147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51848/0 (socket says 172.21.15.46:51848) 2023-12-16T21:43:09.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 -- 172.21.15.46:0/2235807621 learned_addr learned my addr 172.21.15.46:0/2235807621 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9371d9b640 1 --2- 172.21.15.46:0/2235807621 >> v2:172.21.15.161:3300/0 conn(0x7f93740af2e0 0x7f93740ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f937259c640 1 --2- 172.21.15.46:0/2235807621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f937406c920 0x7f93740ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 -- 172.21.15.46:0/2235807621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f937406c920 msgr2=0x7f93740ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 --2- 172.21.15.46:0/2235807621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f937406c920 0x7f93740ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 -- 172.21.15.46:0/2235807621 >> v2:172.21.15.161:3300/0 conn(0x7f93740af2e0 msgr2=0x7f93740ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 --2- 172.21.15.46:0/2235807621 >> v2:172.21.15.161:3300/0 conn(0x7f93740af2e0 0x7f93740ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.143+0000 7f9372d9d640 1 -- 172.21.15.46:0/2235807621 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937406cd00 con 0x7f93740b4430 2023-12-16T21:43:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9372d9d640 1 --2- 172.21.15.46:0/2235807621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f9374147080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f937406cef0 tx=0x7f936c01eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e724e47db19a0402 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f937159a640 1 -- 172.21.15.46:0/2235807621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c00b2d0 con 0x7f93740b4430 2023-12-16T21:43:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f937159a640 1 -- 172.21.15.46:0/2235807621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936c00b430 con 0x7f93740b4430 2023-12-16T21:43:09.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f937159a640 1 -- 172.21.15.46:0/2235807621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0a7b80 con 0x7f93740b4430 2023-12-16T21:43:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- 172.21.15.46:0/2235807621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 msgr2=0x7f9374147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2235807621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f9374147080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f937406cef0 tx=0x7f936c01eab0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- 172.21.15.46:0/2235807621 shutdown_connections 2023-12-16T21:43:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2235807621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f9374147080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2235807621 >> v2:172.21.15.161:3300/0 conn(0x7f93740af2e0 0x7f93740ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2235807621 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f937406c920 0x7f93740ad0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- 172.21.15.46:0/2235807621 >> 172.21.15.46:0/2235807621 conn(0x7f93740b48e0 msgr2=0x7f93740b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- 172.21.15.46:0/2235807621 shutdown_connections 2023-12-16T21:43:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- 172.21.15.46:0/2235807621 wait complete. 2023-12-16T21:43:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 Processor -- start 2023-12-16T21:43:09.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- start start 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f937406c920 0x7f937409b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f93741df670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93741dfbb0 0x7f937413c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937409c2e0 con 0x7f93740b4430 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937409c7a0 con 0x7f937406c920 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9378d4e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937409dd20 con 0x7f93741dfbb0 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f93741df670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f93741df670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51854/0 (socket says 172.21.15.46:51854) 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 -- 172.21.15.46:0/2739679151 learned_addr learned my addr 172.21.15.46:0/2739679151 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9372d9d640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93741dfbb0 0x7f937413c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f937259c640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f937406c920 0x7f937409b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 -- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93741dfbb0 msgr2=0x7f937413c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93741dfbb0 0x7f937413c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 -- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f937406c920 msgr2=0x7f937409b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f937406c920 0x7f937409b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9371d9b640 1 -- 172.21.15.46:0/2739679151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c00bd40 con 0x7f93740b4430 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.147+0000 7f9372d9d640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93741dfbb0 0x7f937413c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9371d9b640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f93741df670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f936800e960 tx=0x7f936800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9362ffd640 1 -- 172.21.15.46:0/2739679151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9368009690 con 0x7f93740b4430 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9362ffd640 1 -- 172.21.15.46:0/2739679151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800c9a0 con 0x7f93740b4430 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9362ffd640 1 -- 172.21.15.46:0/2739679151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800cd10 con 0x7f93740b4430 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93741e0060 con 0x7f93740b4430 2023-12-16T21:43:09.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937409ccd0 con 0x7f93740b4430 2023-12-16T21:43:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93740679e0 con 0x7f93740b4430 2023-12-16T21:43:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9362ffd640 1 -- 172.21.15.46:0/2739679151 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f9368013470 con 0x7f93740b4430 2023-12-16T21:43:09.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9362ffd640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f93540664a0 0x7f9354068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f9362ffd640 1 -- 172.21.15.46:0/2739679151 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9368083ff0 con 0x7f93740b4430 2023-12-16T21:43:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.151+0000 7f937259c640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f93540664a0 0x7f9354068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.155+0000 7f937259c640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f93540664a0 0x7f9354068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f937409d6d0 tx=0x7f9364008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.155+0000 7f9362ffd640 1 -- 172.21.15.46:0/2739679151 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f9368051460 con 0x7f93740b4430 2023-12-16T21:43:09.353 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93740688f0 con 0x7f93740b4430 2023-12-16T21:43:09.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9362ffd640 1 -- 172.21.15.46:0/2739679151 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3352 (secure 0 0 0) 0x7f9368017d30 con 0x7f93740b4430 2023-12-16T21:43:09.354 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:09.354 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4212,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/1195756161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1195756161},{"type":"v1","addr":"172.21.15.46:6833","nonce":1195756161}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:42:40.776790+0000","modified":"2023-12-16T21:42:42.763769+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.161:6835/926962350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":926962350},{"type":"v1","addr":"172.21.15.161:6835","nonce":926962350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-16T21:43:09.355 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f93540664a0 msgr2=0x7f9354068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f93540664a0 0x7f9354068960 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f937409d6d0 tx=0x7f9364008040 comp rx=0 tx=0).stop 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 msgr2=0x7f93741df670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f93741df670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f936800e960 tx=0x7f936800ee20 comp rx=0 tx=0).stop 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 shutdown_connections 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f93540664a0 0x7f9354068960 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f93741dfbb0 0x7f937413c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93740b4430 0x7f93741df670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.357 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 --2- 172.21.15.46:0/2739679151 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f937406c920 0x7f937409b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 >> 172.21.15.46:0/2739679151 conn(0x7f93740b48e0 msgr2=0x7f93740b6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:09.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 shutdown_connections 2023-12-16T21:43:09.358 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.347+0000 7f9378d4e640 1 -- 172.21.15.46:0/2739679151 wait complete. 2023-12-16T21:43:09.374 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.483+0000 7f2af60d8640 1 Processor -- start 2023-12-16T21:43:09.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.483+0000 7f2af60d8640 1 -- start start 2023-12-16T21:43:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2af60d8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af00b70b0 0x7f2af00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2af60d8640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2af00b4860 0x7f2af00b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2af60d8640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f2af0070910 0x7f2af0070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2af60d8640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00b5780 con 0x7f2af00b4860 2023-12-16T21:43:09.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2af60d8640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00b6120 con 0x7f2af0070910 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2af60d8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00b5200 con 0x7f2af00b70b0 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2af00b4860 0x7f2af00b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2af00b4860 0x7f2af00b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51868/0 (socket says 172.21.15.46:51868) 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 -- 172.21.15.46:0/2434454889 learned_addr learned my addr 172.21.15.46:0/2434454889 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aef7fe640 1 --2- 172.21.15.46:0/2434454889 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af00b70b0 0x7f2af00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeffff640 1 --2- 172.21.15.46:0/2434454889 >> v2:172.21.15.161:3300/0 conn(0x7f2af0070910 0x7f2af0070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 -- 172.21.15.46:0/2434454889 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af00b70b0 msgr2=0x7f2af00b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 --2- 172.21.15.46:0/2434454889 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af00b70b0 0x7f2af00b0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 -- 172.21.15.46:0/2434454889 >> v2:172.21.15.161:3300/0 conn(0x7f2af0070910 msgr2=0x7f2af0070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 --2- 172.21.15.46:0/2434454889 >> v2:172.21.15.161:3300/0 conn(0x7f2af0070910 0x7f2af0070d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.487+0000 7f2aeeffd640 1 -- 172.21.15.46:0/2434454889 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00b7490 con 0x7f2af00b4860 2023-12-16T21:43:09.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 --2- 172.21.15.46:0/2434454889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af00b0cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2af0072270 tx=0x7f2ae401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169816ebdbb1bd0c server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aee7fc640 1 -- 172.21.15.46:0/2434454889 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40ab600 con 0x7f2af00b4860 2023-12-16T21:43:09.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aee7fc640 1 -- 172.21.15.46:0/2434454889 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae40a7050 con 0x7f2af00b4860 2023-12-16T21:43:09.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aee7fc640 1 -- 172.21.15.46:0/2434454889 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40ab980 con 0x7f2af00b4860 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- 172.21.15.46:0/2434454889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 msgr2=0x7f2af00b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 --2- 172.21.15.46:0/2434454889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af00b0cb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2af0072270 tx=0x7f2ae401c500 comp rx=0 tx=0).stop 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- 172.21.15.46:0/2434454889 shutdown_connections 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 --2- 172.21.15.46:0/2434454889 >> v2:172.21.15.161:3300/0 conn(0x7f2af0070910 0x7f2af0070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 --2- 172.21.15.46:0/2434454889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af00b0cb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 --2- 172.21.15.46:0/2434454889 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af00b70b0 0x7f2af00b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- 172.21.15.46:0/2434454889 >> 172.21.15.46:0/2434454889 conn(0x7f2af006a980 msgr2=0x7f2af006ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- 172.21.15.46:0/2434454889 shutdown_connections 2023-12-16T21:43:09.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- 172.21.15.46:0/2434454889 wait complete. 2023-12-16T21:43:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 Processor -- start 2023-12-16T21:43:09.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- start start 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af0070910 0x7f2af013b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af013f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af00b70b0 0x7f2af01e1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0141a70 con 0x7f2af00b4860 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0141ce0 con 0x7f2af00b70b0 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2af60d8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af013fb30 con 0x7f2af0070910 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af013f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af013f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51870/0 (socket says 172.21.15.46:51870) 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 -- 172.21.15.46:0/512146767 learned_addr learned my addr 172.21.15.46:0/512146767 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aef7fe640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af0070910 0x7f2af013b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeffff640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af00b70b0 0x7f2af01e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 -- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af0070910 msgr2=0x7f2af013b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af0070910 0x7f2af013b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 -- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af00b70b0 msgr2=0x7f2af01e1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af00b70b0 0x7f2af01e1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 -- 172.21.15.46:0/512146767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae40094d0 con 0x7f2af00b4860 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeffff640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af00b70b0 0x7f2af01e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.491+0000 7f2aeeffd640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af013f1e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ae401abb0 tx=0x7f2ae40192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2ad7fff640 1 -- 172.21.15.46:0/512146767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40b37d0 con 0x7f2af00b4860 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2ad7fff640 1 -- 172.21.15.46:0/512146767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae40b3930 con 0x7f2af00b4860 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af013fe10 con 0x7f2af00b4860 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2ad7fff640 1 -- 172.21.15.46:0/512146767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40b3ca0 con 0x7f2af00b4860 2023-12-16T21:43:09.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af01403d0 con 0x7f2af00b4860 2023-12-16T21:43:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af00679e0 con 0x7f2af00b4860 2023-12-16T21:43:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2ad7fff640 1 -- 172.21.15.46:0/512146767 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2ae40b2070 con 0x7f2af00b4860 2023-12-16T21:43:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2ad7fff640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2adc066470 0x7f2adc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2ad7fff640 1 -- 172.21.15.46:0/512146767 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ae411ebd0 con 0x7f2af00b4860 2023-12-16T21:43:09.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2aef7fe640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2adc066470 0x7f2adc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.495+0000 7f2aef7fe640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2adc066470 0x7f2adc068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000d8d0 tx=0x7f2ae0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.499+0000 7f2ad7fff640 1 -- 172.21.15.46:0/512146767 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2ae411ee60 con 0x7f2af00b4860 2023-12-16T21:43:09.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.655+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2af01438d0 con 0x7f2af00b4860 2023-12-16T21:43:09.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.659+0000 7f2ad7fff640 1 -- 172.21.15.46:0/512146767 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16505 (secure 0 0 0) 0x7f2ae40ec140 con 0x7f2af00b4860 2023-12-16T21:43:09.667 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:09.667 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:42:42.321651+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:42:38.743410+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-16T21:42:39.749094+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2adc066470 msgr2=0x7f2adc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2adc066470 0x7f2adc068930 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ae000d8d0 tx=0x7f2ae0008040 comp rx=0 tx=0).stop 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 msgr2=0x7f2af013f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af013f1e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ae401abb0 tx=0x7f2ae40192a0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 shutdown_connections 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2adc066470 0x7f2adc068930 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af00b70b0 0x7f2af01e1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af00b4860 0x7f2af013f1e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 --2- 172.21.15.46:0/512146767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af0070910 0x7f2af013b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 >> 172.21.15.46:0/512146767 conn(0x7f2af006a980 msgr2=0x7f2af0143d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 shutdown_connections 2023-12-16T21:43:09.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.663+0000 7f2af60d8640 1 -- 172.21.15.46:0/512146767 wait complete. 2023-12-16T21:43:09.686 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-16T21:43:09.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 Processor -- start 2023-12-16T21:43:09.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 -- start start 2023-12-16T21:43:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc2c0ad2f0 0x7fbc2c06b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fbc2c06bbf0 0x7fbc2c06c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c06c580 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b2320 con 0x7fbc2c06bbf0 2023-12-16T21:43:09.799 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc34743640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0affc0 con 0x7fbc2c0b3b00 2023-12-16T21:43:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc324df640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc31cde640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fbc2c0ad2f0 0x7fbc2c06b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc324df640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c0acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:38862/0 (socket says 172.21.15.46:38862) 2023-12-16T21:43:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc324df640 1 -- 172.21.15.46:0/3108038122 learned_addr learned my addr 172.21.15.46:0/3108038122 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc32ce0640 1 --2- 172.21.15.46:0/3108038122 >> v2:172.21.15.161:3300/0 conn(0x7fbc2c06bbf0 0x7fbc2c06c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc31cde640 1 -- 172.21.15.46:0/3108038122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 msgr2=0x7fbc2c0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc31cde640 1 --2- 172.21.15.46:0/3108038122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c0acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc31cde640 1 -- 172.21.15.46:0/3108038122 >> v2:172.21.15.161:3300/0 conn(0x7fbc2c06bbf0 msgr2=0x7fbc2c06c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc31cde640 1 --2- 172.21.15.46:0/3108038122 >> v2:172.21.15.161:3300/0 conn(0x7fbc2c06bbf0 0x7fbc2c06c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc31cde640 1 -- 172.21.15.46:0/3108038122 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0b3ee0 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.791+0000 7fbc324df640 1 --2- 172.21.15.46:0/3108038122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c0acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:09.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc31cde640 1 --2- 172.21.15.46:0/3108038122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c06b6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0b4840 tx=0x7fbc2801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e132a01600b4636 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc314dd640 1 -- 172.21.15.46:0/3108038122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc280ab600 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc314dd640 1 -- 172.21.15.46:0/3108038122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc280a7050 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc314dd640 1 -- 172.21.15.46:0/3108038122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc280ab980 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 -- 172.21.15.46:0/3108038122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 msgr2=0x7fbc2c06b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 --2- 172.21.15.46:0/3108038122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c06b6b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c0b4840 tx=0x7fbc2801c460 comp rx=0 tx=0).stop 2023-12-16T21:43:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 -- 172.21.15.46:0/3108038122 shutdown_connections 2023-12-16T21:43:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 --2- 172.21.15.46:0/3108038122 >> v2:172.21.15.161:3300/0 conn(0x7fbc2c06bbf0 0x7fbc2c06c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 --2- 172.21.15.46:0/3108038122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c06b6b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 --2- 172.21.15.46:0/3108038122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c0acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 -- 172.21.15.46:0/3108038122 >> 172.21.15.46:0/3108038122 conn(0x7fbc2c06a580 msgr2=0x7fbc2c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 -- 172.21.15.46:0/3108038122 shutdown_connections 2023-12-16T21:43:09.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.795+0000 7fbc34743640 1 -- 172.21.15.46:0/3108038122 wait complete. 2023-12-16T21:43:09.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 Processor -- start 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 -- start start 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c06bbf0 0x7fbc2c149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c14a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c14a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0b36d0 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c14b7d0 con 0x7fbc2c06bbf0 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c142d10 con 0x7fbc2c0b3b00 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c14a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c14a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51884/0 (socket says 172.21.15.46:51884) 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 -- 172.21.15.46:0/647337524 learned_addr learned my addr 172.21.15.46:0/647337524 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc32ce0640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c14a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc324df640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c06bbf0 0x7fbc2c149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 -- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 msgr2=0x7fbc2c14a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c14a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 -- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c06bbf0 msgr2=0x7fbc2c149860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c06bbf0 0x7fbc2c149860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 -- 172.21.15.46:0/647337524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24009b30 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc32ce0640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c14a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc324df640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c06bbf0 0x7fbc2c149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc31cde640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c14a160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbc2801cd00 tx=0x7fbc28019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc22ffd640 1 -- 172.21.15.46:0/647337524 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc280abe70 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc22ffd640 1 -- 172.21.15.46:0/647337524 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc280b4430 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc22ffd640 1 -- 172.21.15.46:0/647337524 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc280b4800 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc28009420 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c0b2930 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c0679e0 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc22ffd640 1 -- 172.21.15.46:0/647337524 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fbc280b2040 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc22ffd640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbc18066470 0x7fbc18068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:09.812 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.799+0000 7fbc22ffd640 1 -- 172.21.15.46:0/647337524 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc2811dd40 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.803+0000 7fbc324df640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbc18066470 0x7fbc18068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.803+0000 7fbc324df640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbc18066470 0x7fbc18068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009fd0 tx=0x7fbc2400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:09.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.803+0000 7fbc22ffd640 1 -- 172.21.15.46:0/647337524 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fbc280eb250 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.997 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.991+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbc2c0b3530 con 0x7fbc2c0ad2f0 2023-12-16T21:43:09.997 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-12-16T21:43:09.991+0000 7f80e84dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-16T21:43:10.000 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.991+0000 7fbc22ffd640 1 -- 172.21.15.46:0/647337524 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbc2c0b3530 con 0x7fbc2c0ad2f0 2023-12-16T21:43:10.000 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-16T21:43:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbc18066470 msgr2=0x7fbc18068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbc18066470 0x7fbc18068930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009fd0 tx=0x7fbc2400b040 comp rx=0 tx=0).stop 2023-12-16T21:43:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 msgr2=0x7fbc2c14a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c14a160 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbc2801cd00 tx=0x7fbc28019200 comp rx=0 tx=0).stop 2023-12-16T21:43:10.002 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 shutdown_connections 2023-12-16T21:43:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbc18066470 0x7fbc18068930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc2c0b3b00 0x7fbc2c14a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc2c0ad2f0 0x7fbc2c14a160 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 --2- 172.21.15.46:0/647337524 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc2c06bbf0 0x7fbc2c149860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 >> 172.21.15.46:0/647337524 conn(0x7fbc2c06a580 msgr2=0x7fbc2c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 shutdown_connections 2023-12-16T21:43:10.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:09.995+0000 7fbc34743640 1 -- 172.21.15.46:0/647337524 wait complete. 2023-12-16T21:43:10.012 INFO:tasks.ceph.mds.b.smithi161.stderr: -1007> 2023-12-16T21:42:45.328+0000 7f3edb8f1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-16T21:43:10.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-16T21:43:10.042 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-12-16T21:43:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f5874ee2640 1 Processor -- start 2023-12-16T21:43:10.143 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f5874ee2640 1 -- start start 2023-12-16T21:43:10.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f5874ee2640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f587006c8b0 0x7f58700b4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:10.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f5874ee2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f58700af300 0x7f58700ace20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:10.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f5874ee2640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58700b02e0 con 0x7f58700ae750 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f5874ee2640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58700aded0 con 0x7f587006c8b0 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f5874ee2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587006cc90 con 0x7f58700af300 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f58700af300 0x7f58700ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.135+0000 7f586e59c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f587006c8b0 0x7f58700b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586e59c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f587006c8b0 0x7f58700b4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:40118/0 (socket says 172.21.15.46:40118) 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f58700af300 0x7f58700ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:38874/0 (socket says 172.21.15.46:38874) 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586e59c640 1 -- 172.21.15.46:0/2826529561 learned_addr learned my addr 172.21.15.46:0/2826529561 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586dd9b640 1 -- 172.21.15.46:0/2826529561 >> v2:172.21.15.161:3300/0 conn(0x7f587006c8b0 msgr2=0x7f58700b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586dd9b640 1 --2- 172.21.15.46:0/2826529561 >> v2:172.21.15.161:3300/0 conn(0x7f587006c8b0 0x7f58700b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586dd9b640 1 -- 172.21.15.46:0/2826529561 >> v1:172.21.15.161:6789/0 conn(0x7f58700ae750 legacy=0x7f58700ad360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:10.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586dd9b640 1 -- 172.21.15.46:0/2826529561 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700b05c0 con 0x7f58700af300 2023-12-16T21:43:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586dd9b640 1 --2- 172.21.15.46:0/2826529561 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f58700af300 0x7f58700ace20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58580097c0 tx=0x7f585801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4424f63f1cb2b7b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586d59a640 1 -- 172.21.15.46:0/2826529561 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58580ab600 con 0x7f58700af300 2023-12-16T21:43:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586d59a640 1 -- 172.21.15.46:0/2826529561 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58580a7050 con 0x7f58700af300 2023-12-16T21:43:10.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.139+0000 7f586d59a640 1 -- 172.21.15.46:0/2826529561 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58580ab980 con 0x7f58700af300 2023-12-16T21:43:10.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- 172.21.15.46:0/2826529561 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f58700af300 msgr2=0x7f58700ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:10.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 --2- 172.21.15.46:0/2826529561 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f58700af300 0x7f58700ace20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58580097c0 tx=0x7f585801c330 comp rx=0 tx=0).stop 2023-12-16T21:43:10.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- 172.21.15.46:0/2826529561 shutdown_connections 2023-12-16T21:43:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 --2- 172.21.15.46:0/2826529561 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f58700af300 0x7f58700ace20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 --2- 172.21.15.46:0/2826529561 >> v2:172.21.15.161:3300/0 conn(0x7f587006c8b0 0x7f58700b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- 172.21.15.46:0/2826529561 >> 172.21.15.46:0/2826529561 conn(0x7f58700b4900 msgr2=0x7f58700b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- 172.21.15.46:0/2826529561 shutdown_connections 2023-12-16T21:43:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- 172.21.15.46:0/2826529561 wait complete. 2023-12-16T21:43:10.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 Processor -- start 2023-12-16T21:43:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- start start 2023-12-16T21:43:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f587006c8b0 0x7f58700b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58700ae750 0x7f58701df9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58700af300 0x7f58701dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58700b3930 con 0x7f58700ae750 2023-12-16T21:43:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5870140c80 con 0x7f58700af300 2023-12-16T21:43:10.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58700b3690 con 0x7f587006c8b0 2023-12-16T21:43:10.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58700ae750 0x7f58701df9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:10.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58700ae750 0x7f58701df9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51898/0 (socket says 172.21.15.46:51898) 2023-12-16T21:43:10.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 -- 172.21.15.46:0/738634674 learned_addr learned my addr 172.21.15.46:0/738634674 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:10.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586e59c640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f587006c8b0 0x7f58700b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586ed9d640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58700af300 0x7f58701dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 -- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f587006c8b0 msgr2=0x7f58700b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f587006c8b0 0x7f58700b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 -- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58700af300 msgr2=0x7f58701dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58700af300 0x7f58701dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 -- 172.21.15.46:0/738634674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5858009420 con 0x7f58700ae750 2023-12-16T21:43:10.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f586dd9b640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58700ae750 0x7f58701df9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f585801a9e0 tx=0x7f58580190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:10.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5866ffd640 1 -- 172.21.15.46:0/738634674 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58580b4400 con 0x7f58700ae750 2023-12-16T21:43:10.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5866ffd640 1 -- 172.21.15.46:0/738634674 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58580b4580 con 0x7f58700ae750 2023-12-16T21:43:10.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5870141de0 con 0x7f58700ae750 2023-12-16T21:43:10.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5866ffd640 1 -- 172.21.15.46:0/738634674 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58580b4930 con 0x7f58700ae750 2023-12-16T21:43:10.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.143+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5870141b00 con 0x7f58700ae750 2023-12-16T21:43:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.147+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5870140740 con 0x7f58700ae750 2023-12-16T21:43:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.147+0000 7f5866ffd640 1 -- 172.21.15.46:0/738634674 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f58580b2040 con 0x7f58700ae750 2023-12-16T21:43:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.147+0000 7f5866ffd640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5850066470 0x7f5850068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.147+0000 7f5866ffd640 1 -- 172.21.15.46:0/738634674 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f585811dcd0 con 0x7f58700ae750 2023-12-16T21:43:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.147+0000 7f586e59c640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5850066470 0x7f5850068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:10.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.147+0000 7f586e59c640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5850066470 0x7f5850068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5860009ae0 tx=0x7f5860008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:10.158 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.151+0000 7f5866ffd640 1 -- 172.21.15.46:0/738634674 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f58580eb180 con 0x7f58700ae750 2023-12-16T21:43:10.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.339+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5870068960 con 0x7f58700ae750 2023-12-16T21:43:11.003 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.995+0000 7f5866ffd640 1 -- 172.21.15.46:0/738634674 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f58580ee7a0 con 0x7f58700ae750 2023-12-16T21:43:11.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5850066470 msgr2=0x7f5850068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5850066470 0x7f5850068930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5860009ae0 tx=0x7f5860008040 comp rx=0 tx=0).stop 2023-12-16T21:43:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58700ae750 msgr2=0x7f58701df9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58700ae750 0x7f58701df9b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f585801a9e0 tx=0x7f58580190d0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 shutdown_connections 2023-12-16T21:43:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5850066470 0x7f5850068930 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f58700af300 0x7f58701dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58700ae750 0x7f58701df9b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 --2- 172.21.15.46:0/738634674 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f587006c8b0 0x7f58700b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 >> 172.21.15.46:0/738634674 conn(0x7f58700b4900 msgr2=0x7f58700b6460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 shutdown_connections 2023-12-16T21:43:11.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:10.999+0000 7f5874ee2640 1 -- 172.21.15.46:0/738634674 wait complete. 2023-12-16T21:43:11.027 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-16T21:43:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f44948d5640 1 Processor -- start 2023-12-16T21:43:11.160 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f44948d5640 1 -- start start 2023-12-16T21:43:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f44948d5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f449006c920 0x7f4490071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f44948d5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900ad750 con 0x7f4490072200 2023-12-16T21:43:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f44948d5640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44900adff0 con 0x7f4490149e60 2023-12-16T21:43:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f44948d5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f449006cd00 con 0x7f449006c920 2023-12-16T21:43:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f448e59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f449006c920 0x7f4490071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:11.161 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f448dd9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4490072200 0x7f44900b3e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46550/0 (socket says 172.21.15.46:46550) 2023-12-16T21:43:11.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.151+0000 7f448dd9b640 1 -- 172.21.15.46:0/948076310 learned_addr learned my addr 172.21.15.46:0/948076310 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:11.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f448e59c640 1 -- 172.21.15.46:0/948076310 >> v1:172.21.15.161:6789/0 conn(0x7f4490149e60 legacy=0x7f449014a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:11.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f448e59c640 1 -- 172.21.15.46:0/948076310 >> v1:172.21.15.46:6789/0 conn(0x7f4490072200 legacy=0x7f44900b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:11.162 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f448e59c640 1 -- 172.21.15.46:0/948076310 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900ad9a0 con 0x7f449006c920 2023-12-16T21:43:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f448e59c640 1 --2- 172.21.15.46:0/948076310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490071cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f44800097f0 tx=0x7f448001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90fc648b109c92f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f448d59a640 1 -- 172.21.15.46:0/948076310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44900b4730 con 0x7f449006c920 2023-12-16T21:43:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f448d59a640 1 -- 172.21.15.46:0/948076310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44800a7050 con 0x7f449006c920 2023-12-16T21:43:11.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f448d59a640 1 -- 172.21.15.46:0/948076310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800ab820 con 0x7f449006c920 2023-12-16T21:43:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f44948d5640 1 -- 172.21.15.46:0/948076310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 msgr2=0x7f4490071cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f44948d5640 1 --2- 172.21.15.46:0/948076310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490071cc0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f44800097f0 tx=0x7f448001c3b0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f44948d5640 1 -- 172.21.15.46:0/948076310 shutdown_connections 2023-12-16T21:43:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f44948d5640 1 --2- 172.21.15.46:0/948076310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490071cc0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f44948d5640 1 -- 172.21.15.46:0/948076310 >> 172.21.15.46:0/948076310 conn(0x7f44900b4890 msgr2=0x7f44900b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f44948d5640 1 -- 172.21.15.46:0/948076310 shutdown_connections 2023-12-16T21:43:11.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.155+0000 7f44948d5640 1 -- 172.21.15.46:0/948076310 wait complete. 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 Processor -- start 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 -- start start 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4490072200 0x7f44901df310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4490149e60 0x7f44901df850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44901deec0 con 0x7f449006c920 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4490146dc0 con 0x7f4490149e60 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44901e2dd0 con 0x7f4490072200 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490139b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51918/0 (socket says 172.21.15.46:51918) 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 -- 172.21.15.46:0/4090959781 learned_addr learned my addr 172.21.15.46:0/4090959781 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:11.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448dd9b640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4490072200 0x7f44901df310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448ed9d640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4490149e60 0x7f44901df850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 -- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4490072200 msgr2=0x7f44901df310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4490072200 0x7f44901df310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 -- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4490149e60 msgr2=0x7f44901df850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4490149e60 0x7f44901df850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 -- 172.21.15.46:0/4090959781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480009420 con 0x7f449006c920 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448ed9d640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4490149e60 0x7f44901df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f448e59c640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490139b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4480018fc0 tx=0x7f4480019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f447effd640 1 -- 172.21.15.46:0/4090959781 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800b4410 con 0x7f449006c920 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f447effd640 1 -- 172.21.15.46:0/4090959781 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44800b4570 con 0x7f449006c920 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44901e30b0 con 0x7f449006c920 2023-12-16T21:43:11.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f447effd640 1 -- 172.21.15.46:0/4090959781 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44800b4920 con 0x7f449006c920 2023-12-16T21:43:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4490146920 con 0x7f449006c920 2023-12-16T21:43:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.159+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44901df160 con 0x7f449006c920 2023-12-16T21:43:11.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.163+0000 7f447effd640 1 -- 172.21.15.46:0/4090959781 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f44800b2040 con 0x7f449006c920 2023-12-16T21:43:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.163+0000 7f447effd640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4470066470 0x7f4470068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.163+0000 7f447effd640 1 -- 172.21.15.46:0/4090959781 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f448011dfd0 con 0x7f449006c920 2023-12-16T21:43:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.163+0000 7f448dd9b640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4470066470 0x7f4470068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.163+0000 7f448dd9b640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4470066470 0x7f4470068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f44901dfff0 tx=0x7f4484009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.167+0000 7f447effd640 1 -- 172.21.15.46:0/4090959781 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f44800eb5a0 con 0x7f449006c920 2023-12-16T21:43:11.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:11.323+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f44900689d0 con 0x7f449006c920 2023-12-16T21:43:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.003+0000 7f447effd640 1 -- 172.21.15.46:0/4090959781 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f44800eebc0 con 0x7f449006c920 2023-12-16T21:43:12.009 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-12-16T21:43:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.007+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4470066470 msgr2=0x7f4470068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.007+0000 7f44948d5640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4470066470 0x7f4470068930 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f44901dfff0 tx=0x7f4484009040 comp rx=0 tx=0).stop 2023-12-16T21:43:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.007+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 msgr2=0x7f4490139b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:12.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.007+0000 7f44948d5640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490139b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4480018fc0 tx=0x7f4480019150 comp rx=0 tx=0).stop 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 shutdown_connections 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4470066470 0x7f4470068930 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4490149e60 0x7f44901df850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4490072200 0x7f44901df310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 --2- 172.21.15.46:0/4090959781 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f449006c920 0x7f4490139b40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 >> 172.21.15.46:0/4090959781 conn(0x7f44900b4890 msgr2=0x7f44900b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 shutdown_connections 2023-12-16T21:43:12.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.011+0000 7f44948d5640 1 -- 172.21.15.46:0/4090959781 wait complete. 2023-12-16T21:43:12.042 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-16T21:43:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c9260640 1 Processor -- start 2023-12-16T21:43:12.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c9260640 1 -- start start 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c9260640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd3c40b4a10 0x7fd3c4146700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c9260640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c40b87e0 con 0x7fd3c40bc870 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c9260640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c40b3870 con 0x7fd3c4140b90 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c9260640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c40bcc50 con 0x7fd3c40b4a10 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c2d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd3c40bc870 0x7fd3c4140480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46552/0 (socket says 172.21.15.46:46552) 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.159+0000 7fd3c2d9d640 1 -- 172.21.15.46:0/2438139767 learned_addr learned my addr 172.21.15.46:0/2438139767 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 --2- 172.21.15.46:0/2438139767 >> v2:172.21.15.161:3300/0 conn(0x7fd3c40b4a10 0x7fd3c4146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301469128 0 0) 0x7fd3b40014a0 con 0x7fd3c40bc870 2023-12-16T21:43:12.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3ac003530 con 0x7fd3c40bc870 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1625623231 0 0) 0x7fd3ac003530 con 0x7fd3c40bc870 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 >> v2:172.21.15.161:3300/0 conn(0x7fd3c40b4a10 msgr2=0x7fd3c4146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 --2- 172.21.15.46:0/2438139767 >> v2:172.21.15.161:3300/0 conn(0x7fd3c40b4a10 0x7fd3c4146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 >> v1:172.21.15.161:6789/0 conn(0x7fd3c4140b90 legacy=0x7fd3c414f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c40b3b10 con 0x7fd3c40bc870 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fd3b4001b90 con 0x7fd3c40bc870 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 >> v1:172.21.15.46:6789/0 conn(0x7fd3c40bc870 legacy=0x7fd3c4140480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3ac003980 0x7fd3ac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac0047d0 0x7fd3ac004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac005390 0x7fd3ac00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c40bcc50 con 0x7fd3ac005390 2023-12-16T21:43:12.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac00c220 con 0x7fd3ac003980 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac00c3e0 con 0x7fd3ac0047d0 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac005390 0x7fd3ac00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c2d9d640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3ac003980 0x7fd3ac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c259c640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac0047d0 0x7fd3ac004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 -- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac0047d0 msgr2=0x7fd3ac004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac0047d0 0x7fd3ac004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 -- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3ac003980 msgr2=0x7fd3ac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3ac003980 0x7fd3ac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 -- 172.21.15.46:0/2438139767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac003530 con 0x7fd3ac005390 2023-12-16T21:43:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c2d9d640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3ac003980 0x7fd3ac003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:12.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c259c640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac0047d0 0x7fd3ac004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 -- 172.21.15.46:0/2438139767 reap_dead start 2023-12-16T21:43:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c359e640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac005390 0x7fd3ac00b990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00ba30 tx=0x7fd3bc01c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7ac1bc76a23520 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc004020 con 0x7fd3ac005390 2023-12-16T21:43:12.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc0041c0 con 0x7fd3ac005390 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.163+0000 7fd3c1d9b640 1 -- 172.21.15.46:0/2438139767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc003620 con 0x7fd3ac005390 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- 172.21.15.46:0/2438139767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac005390 msgr2=0x7fd3ac00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac005390 0x7fd3ac00b990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc00ba30 tx=0x7fd3bc01c600 comp rx=0 tx=0).stop 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- 172.21.15.46:0/2438139767 shutdown_connections 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2438139767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3ac005390 0x7fd3ac00b990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- 172.21.15.46:0/2438139767 >> 172.21.15.46:0/2438139767 conn(0x7fd3c40723e0 msgr2=0x7fd3c40bbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- 172.21.15.46:0/2438139767 shutdown_connections 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- 172.21.15.46:0/2438139767 wait complete. 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 Processor -- start 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- start start 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac005390 0x7fd3c413d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c41e2770 0x7fd3c4137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c412ebf0 0x7fd3c41df200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c413e380 con 0x7fd3c41e2770 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c41e2d20 con 0x7fd3c412ebf0 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c41e3000 con 0x7fd3ac005390 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c41e2770 0x7fd3c4137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c41e2770 0x7fd3c4137ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51942/0 (socket says 172.21.15.46:51942) 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 -- 172.21.15.46:0/2198605515 learned_addr learned my addr 172.21.15.46:0/2198605515 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c2d9d640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac005390 0x7fd3c413d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c359e640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c412ebf0 0x7fd3c41df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 -- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac005390 msgr2=0x7fd3c413d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac005390 0x7fd3c413d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 -- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c412ebf0 msgr2=0x7fd3c41df200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c412ebf0 0x7fd3c41df200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 -- 172.21.15.46:0/2198605515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c41e32e0 con 0x7fd3c41e2770 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c2d9d640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac005390 0x7fd3c413d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:12.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c359e640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c412ebf0 0x7fd3c41df200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c259c640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c41e2770 0x7fd3c4137ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800d540 tx=0x7fd3b800da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3ab7fe640 1 -- 172.21.15.46:0/2198605515 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b8005480 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3ab7fe640 1 -- 172.21.15.46:0/2198605515 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3b8003620 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c413d840 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3ab7fe640 1 -- 172.21.15.46:0/2198605515 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3b80039b0 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.167+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c412f080 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.171+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c40689d0 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.171+0000 7fd3ab7fe640 1 -- 172.21.15.46:0/2198605515 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fd3b8003b10 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.171+0000 7fd3ab7fe640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd3b00664a0 0x7fd3b0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.171+0000 7fd3ab7fe640 1 -- 172.21.15.46:0/2198605515 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fd3b8083b80 con 0x7fd3c41e2770 2023-12-16T21:43:12.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.171+0000 7fd3c2d9d640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd3b00664a0 0x7fd3b0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:12.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.171+0000 7fd3c2d9d640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd3b00664a0 0x7fd3b0068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4001360 tx=0x7fd3b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:12.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.175+0000 7fd3ab7fe640 1 -- 172.21.15.46:0/2198605515 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fd3b80511f0 con 0x7fd3c41e2770 2023-12-16T21:43:12.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:12.323+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3c4145fc0 con 0x7fd3c41e2770 2023-12-16T21:43:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.011+0000 7fd3ab7fe640 1 -- 172.21.15.46:0/2198605515 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd3b8001240 con 0x7fd3c41e2770 2023-12-16T21:43:13.019 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd3b00664a0 msgr2=0x7fd3b0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd3b00664a0 0x7fd3b0068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4001360 tx=0x7fd3b4009040 comp rx=0 tx=0).stop 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c41e2770 msgr2=0x7fd3c4137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c41e2770 0x7fd3c4137ab0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800d540 tx=0x7fd3b800da00 comp rx=0 tx=0).stop 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 shutdown_connections 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd3b00664a0 0x7fd3b0068960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd3c412ebf0 0x7fd3c41df200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3c41e2770 0x7fd3c4137ab0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 --2- 172.21.15.46:0/2198605515 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd3ac005390 0x7fd3c413d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 >> 172.21.15.46:0/2198605515 conn(0x7fd3c40723e0 msgr2=0x7fd3c4152020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 shutdown_connections 2023-12-16T21:43:13.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.015+0000 7fd3c9260640 1 -- 172.21.15.46:0/2198605515 wait complete. 2023-12-16T21:43:13.048 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-16T21:43:13.048 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-16T21:43:13.048 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-16T21:43:13.048 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:43:13.048 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-16T21:43:13.048 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-16T21:43:13.049 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-16T21:43:13.049 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:43:13.063 INFO:tasks.ceph.mds.a:Stopped 2023-12-16T21:43:13.063 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-16T21:43:13.067 INFO:tasks.ceph.mds.b:Stopped 2023-12-16T21:43:13.067 DEBUG:teuthology.orchestra.run.smithi161:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-16T21:43:13.107 INFO:tasks.ceph.mds.a:Started 2023-12-16T21:43:13.107 DEBUG:teuthology.parallel:result is None 2023-12-16T21:43:13.114 INFO:tasks.ceph.mds.b:Started 2023-12-16T21:43:13.114 DEBUG:teuthology.parallel:result is None 2023-12-16T21:43:13.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-16T21:43:13.184 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-16T21:43:13.193 INFO:tasks.ceph.mds.b.smithi161.stdout:starting mds.b at 2023-12-16T21:43:13.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.231+0000 7fe8c993d640 1 Processor -- start 2023-12-16T21:43:13.241 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c993d640 1 -- start start 2023-12-16T21:43:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c993d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 0x7fe8c40b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c993d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40b5220 con 0x7fe8c41401d0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c993d640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40bccd0 con 0x7fe8c40bc8f0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c993d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40b3c00 con 0x7fe8c4146e30 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c37fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe8c41401d0 0x7fe8c4140660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46580/0 (socket says 172.21.15.46:46580) 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c37fe640 1 -- 172.21.15.46:0/215142788 learned_addr learned my addr 172.21.15.46:0/215142788 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c27fc640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 0x7fe8c40b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124757281 0 0) 0x7fe8bc0014a0 con 0x7fe8c41401d0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8ac003530 con 0x7fe8c41401d0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 <== mon.1 v1:172.21.15.161:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445888593 0 0) 0x7fe8c40bccd0 con 0x7fe8c40bc8f0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 --> v1:172.21.15.161:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8bc0014a0 con 0x7fe8c40bc8f0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033617777 0 0) 0x7fe8ac003530 con 0x7fe8c41401d0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 msgr2=0x7fe8c40b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 0x7fe8c40b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 >> v1:172.21.15.161:6789/0 conn(0x7fe8c40bc8f0 legacy=0x7fe8c4146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40b3ee0 con 0x7fe8c41401d0 2023-12-16T21:43:13.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fe8bc001b90 con 0x7fe8c41401d0 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 >> v1:172.21.15.46:6789/0 conn(0x7fe8c41401d0 legacy=0x7fe8c4140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8ac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 0x7fe8ac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac004d10 0x7fe8ac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40b3c00 con 0x7fe8ac003b40 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac00ba40 con 0x7fe8ac004d10 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ac00bc00 con 0x7fe8c4146e30 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8ac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c27fc640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 0x7fe8ac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c37fe640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac004d10 0x7fe8ac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 msgr2=0x7fe8ac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 0x7fe8ac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac004d10 msgr2=0x7fe8ac005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:13.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac004d10 0x7fe8ac005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/215142788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ac003530 con 0x7fe8ac003b40 2023-12-16T21:43:13.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c27fc640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c4146e30 0x7fe8ac0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:13.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.235+0000 7fe8c37fe640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8ac004d10 0x7fe8ac005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8ac003f40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40018d0 tx=0x7fe8b401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02ccbbd5938a5c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40052d0 con 0x7fe8ac003b40 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b4005450 con 0x7fe8ac003b40 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c1ffb640 1 -- 172.21.15.46:0/215142788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4003840 con 0x7fe8ac003b40 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- 172.21.15.46:0/215142788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 msgr2=0x7fe8ac003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 --2- 172.21.15.46:0/215142788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8ac003f40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe8b40018d0 tx=0x7fe8b401c560 comp rx=0 tx=0).stop 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c37fe640 1 -- 172.21.15.46:0/215142788 reap_dead start 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- 172.21.15.46:0/215142788 shutdown_connections 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- 172.21.15.46:0/215142788 >> 172.21.15.46:0/215142788 conn(0x7fe8c40722b0 msgr2=0x7fe8c40bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- 172.21.15.46:0/215142788 shutdown_connections 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- 172.21.15.46:0/215142788 wait complete. 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 Processor -- start 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- start start 2023-12-16T21:43:13.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8c4150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8c41e2b70 0x7fe8c41df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c41e2ff0 0x7fe8c41df9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c41e26e0 con 0x7fe8ac003b40 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c4148c60 con 0x7fe8c41e2b70 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c4143f30 con 0x7fe8c41e2ff0 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8c4150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8c4150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51984/0 (socket says 172.21.15.46:51984) 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/142170332 learned_addr learned my addr 172.21.15.46:0/142170332 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c27fc640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8c41e2b70 0x7fe8c41df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c37fe640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c41e2ff0 0x7fe8c41df9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c41e2ff0 msgr2=0x7fe8c41df9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c41e2ff0 0x7fe8c41df9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8c41e2b70 msgr2=0x7fe8c41df4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8c41e2b70 0x7fe8c41df4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:13.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 -- 172.21.15.46:0/142170332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c41e23b0 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c27fc640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8c41e2b70 0x7fe8c41df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c2ffd640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8c4150680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe8b401c390 tx=0x7fe8ac005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8abfff640 1 -- 172.21.15.46:0/142170332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b40050d0 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8abfff640 1 -- 172.21.15.46:0/142170332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b40a6070 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8abfff640 1 -- 172.21.15.46:0/142170332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4003c20 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c4148eb0 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.239+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c413dbb0 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.243+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c413dbb0 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.243+0000 7fe8abfff640 1 -- 172.21.15.46:0/142170332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fe8b40c2460 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.243+0000 7fe8abfff640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe8b0066470 0x7fe8b0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.243+0000 7fe8abfff640 1 -- 172.21.15.46:0/142170332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe8b411de90 con 0x7fe8ac003b40 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.243+0000 7fe8c27fc640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe8b0066470 0x7fe8b0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:13.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.243+0000 7fe8c27fc640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe8b0066470 0x7fe8b0068930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe8c413c2d0 tx=0x7fe8b8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:13.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.247+0000 7fe8abfff640 1 -- 172.21.15.46:0/142170332 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fe8b40eb6a0 con 0x7fe8ac003b40 2023-12-16T21:43:13.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:13.419+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe8c413dbb0 con 0x7fe8ac003b40 2023-12-16T21:43:14.040 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.031+0000 7fe8abfff640 1 -- 172.21.15.46:0/142170332 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8b40eecc0 con 0x7fe8ac003b40 2023-12-16T21:43:14.040 INFO:teuthology.orchestra.run.smithi046.stderr: removed all blocklist entries 2023-12-16T21:43:14.043 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe8b0066470 msgr2=0x7fe8b0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe8b0066470 0x7fe8b0068930 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe8c413c2d0 tx=0x7fe8b8008040 comp rx=0 tx=0).stop 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 msgr2=0x7fe8c4150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8c4150680 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe8b401c390 tx=0x7fe8ac005800 comp rx=0 tx=0).stop 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 shutdown_connections 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe8b0066470 0x7fe8b0068930 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe8c41e2ff0 0x7fe8c41df9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe8c41e2b70 0x7fe8c41df4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 --2- 172.21.15.46:0/142170332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe8ac003b40 0x7fe8c4150680 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 >> 172.21.15.46:0/142170332 conn(0x7fe8c40722b0 msgr2=0x7fe8c413f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 shutdown_connections 2023-12-16T21:43:14.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.035+0000 7fe8c993d640 1 -- 172.21.15.46:0/142170332 wait complete. 2023-12-16T21:43:14.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-16T21:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.183+0000 7f0169dda640 1 Processor -- start 2023-12-16T21:43:14.192 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.183+0000 7f0169dda640 1 -- start start 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f0169dda640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01640af2e0 0x7f0164147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f0169dda640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01640af770 con 0x7f016406c920 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f0169dda640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01640aeea0 con 0x7f01640ad7c0 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f0169dda640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01640afc20 con 0x7f01640af2e0 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f016406c920 0x7f01640ad0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46584/0 (socket says 172.21.15.46:46584) 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 -- 172.21.15.46:0/4161218179 learned_addr learned my addr 172.21.15.46:0/4161218179 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f0162ffd640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01640af2e0 0x7f0164147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247139862 0 0) 0x7f01640af770 con 0x7f016406c920 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f014c003530 con 0x7f016406c920 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1197916236 0 0) 0x7f014c003530 con 0x7f016406c920 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01640af2e0 msgr2=0x7f0164147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f01640af2e0 0x7f0164147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 >> v1:172.21.15.161:6789/0 conn(0x7f01640ad7c0 legacy=0x7f016414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016406cd00 con 0x7f016406c920 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f0154001a00 con 0x7f016406c920 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 >> v1:172.21.15.46:6789/0 conn(0x7f016406c920 legacy=0x7f01640ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f014c0047d0 0x7f014c004c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c005150 0x7f014c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01640afc20 con 0x7f014c0039b0 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014c00bfe0 con 0x7f014c005150 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f014c00c1a0 con 0x7f014c0047d0 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f0162ffd640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f014c0047d0 0x7f014c004c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f0163fff640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c005150 0x7f014c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 -- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f014c0047d0 msgr2=0x7f014c004c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f014c0047d0 0x7f014c004c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 -- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c005150 msgr2=0x7f014c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f014c005150 0x7f014c00b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.197 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 -- 172.21.15.46:0/4161218179 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c003530 con 0x7f014c0039b0 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.187+0000 7f01637fe640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f015400a830 tx=0x7f015401d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e49bed5bdfa75a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01540072d0 con 0x7f014c0039b0 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01540a8070 con 0x7f014c0039b0 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01627fc640 1 -- 172.21.15.46:0/4161218179 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01540076b0 con 0x7f014c0039b0 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- 172.21.15.46:0/4161218179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 msgr2=0x7f014c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 --2- 172.21.15.46:0/4161218179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f014c003dd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f015400a830 tx=0x7f015401d310 comp rx=0 tx=0).stop 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0163fff640 1 -- 172.21.15.46:0/4161218179 reap_dead start 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- 172.21.15.46:0/4161218179 shutdown_connections 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- 172.21.15.46:0/4161218179 >> 172.21.15.46:0/4161218179 conn(0x7f01640b48e0 msgr2=0x7f01640b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- 172.21.15.46:0/4161218179 shutdown_connections 2023-12-16T21:43:14.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- 172.21.15.46:0/4161218179 wait complete. 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 Processor -- start 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- start start 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f016413ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016413b9d0 0x7f016413c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f016413cba0 0x7f01641dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01641397f0 con 0x7f014c0039b0 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016413c1d0 con 0x7f016413b9d0 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016413b050 con 0x7f016413cba0 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f016413ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f016413ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52000/0 (socket says 172.21.15.46:52000) 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 -- 172.21.15.46:0/1717687535 learned_addr learned my addr 172.21.15.46:0/1717687535 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0162ffd640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016413b9d0 0x7f016413c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.201 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0163fff640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f016413cba0 0x7f01641dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 -- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f016413cba0 msgr2=0x7f01641dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f016413cba0 0x7f01641dfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 -- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016413b9d0 msgr2=0x7f016413c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016413b9d0 0x7f016413c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 -- 172.21.15.46:0/1717687535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f016413d020 con 0x7f014c0039b0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f01637fe640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f016413ab10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0154007040 tx=0x7f014c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f014bfff640 1 -- 172.21.15.46:0/1717687535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01540017c0 con 0x7f014c0039b0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f014bfff640 1 -- 172.21.15.46:0/1717687535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0154001920 con 0x7f014c0039b0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f016413b2b0 con 0x7f014c0039b0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f014bfff640 1 -- 172.21.15.46:0/1717687535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0154004d70 con 0x7f014c0039b0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.191+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f016413e050 con 0x7f014c0039b0 2023-12-16T21:43:14.202 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.195+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01640679e0 con 0x7f014c0039b0 2023-12-16T21:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.195+0000 7f014bfff640 1 -- 172.21.15.46:0/1717687535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f01540aa030 con 0x7f014c0039b0 2023-12-16T21:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.195+0000 7f014bfff640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f01500664c0 0x7f0150068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.195+0000 7f014bfff640 1 -- 172.21.15.46:0/1717687535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f015411d900 con 0x7f014c0039b0 2023-12-16T21:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.195+0000 7f0162ffd640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f01500664c0 0x7f0150068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.203 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.195+0000 7f0162ffd640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f01500664c0 0x7f0150068980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f016409de40 tx=0x7f0158008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.206 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.199+0000 7f014bfff640 1 -- 172.21.15.46:0/1717687535 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f01540eb140 con 0x7f014c0039b0 2023-12-16T21:43:14.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.407+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0164139650 con 0x7f014c0039b0 2023-12-16T21:43:14.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f014bfff640 1 -- 172.21.15.46:0/1717687535 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+5207 (secure 0 0 0) 0x7f0164139650 con 0x7f014c0039b0 2023-12-16T21:43:14.417 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:14.417 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "auth_dump": [ 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.a", 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCSGX5lBefnKRAAnNh84aNm6CBfCEA6sfo8bw==", 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-16T21:43:14.418 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mds.b", 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCSGX5lrxzsLxAANfoHUyBQNab5129OdBJH0Q==", 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.0", 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCUGX5lMAwnAxAADVv9fIbJsTxnHtiyQxarSw==", 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.1", 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCXGX5llUz8ChAAZJexX0tCjCLl6tVI/yArWQ==", 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.420 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.2", 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCaGX5lcDSaExAAgAnW1qGMIg5PVqi4AW0Gwg==", 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.3", 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCdGX5lhpIiHBAAiNCrUEeKg81BaM7c7PlTIw==", 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.4", 2023-12-16T21:43:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCjGX5lGB25JBAAo+/p53vX+K7QmkeTH3ImcQ==", 2023-12-16T21:43:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.423 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.424 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.5", 2023-12-16T21:43:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCmGX5l9tIsKxAA2+qYGxVKxakKzPLUQFX3rg==", 2023-12-16T21:43:14.425 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.6", 2023-12-16T21:43:14.427 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCpGX5l+grMNRAAFoWyoAatVVmOueZ/qFVosA==", 2023-12-16T21:43:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.428 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "osd.7", 2023-12-16T21:43:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCtGX5l8BphAhAAcENaFhTC3yX5ilka5mhS5g==", 2023-12-16T21:43:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.429 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow profile osd", 2023-12-16T21:43:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile osd", 2023-12-16T21:43:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.430 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.0", 2023-12-16T21:43:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCSGX5lKFNZNRAAtR1lv++0meNbUxdnBSKAGg==", 2023-12-16T21:43:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow", 2023-12-16T21:43:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow r", 2023-12-16T21:43:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow rw", 2023-12-16T21:43:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow rwx" 2023-12-16T21:43:14.431 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.admin", 2023-12-16T21:43:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCSGX5lhbEoExAAlAn2eLb5iSwapske00BBow==", 2023-12-16T21:43:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.432 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-16T21:43:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr": "allow *", 2023-12-16T21:43:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow *", 2023-12-16T21:43:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.433 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mds", 2023-12-16T21:43:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC3GX5lDOKVJBAAo+rNwGRmgnYYjinVZDzLQw==", 2023-12-16T21:43:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mds" 2023-12-16T21:43:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.434 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-mgr", 2023-12-16T21:43:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC3GX5lmAWWJBAA9vJUSdb2IUJ8L95hnjO2FQ==", 2023-12-16T21:43:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-16T21:43:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.435 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-osd", 2023-12-16T21:43:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC3GX5lLR2WJBAAF5G6/8bS0Ro0mxgzfg3rFA==", 2023-12-16T21:43:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-osd" 2023-12-16T21:43:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.436 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd", 2023-12-16T21:43:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC3GX5lCjSWJBAAt3/cG2EMXHBngaZD9/IDsg==", 2023-12-16T21:43:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-16T21:43:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-16T21:43:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC3GX5l5EqWJBAAO9KnxjnOq8Oinzd3w2Utlg==", 2023-12-16T21:43:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-16T21:43:14.438 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "client.bootstrap-rgw", 2023-12-16T21:43:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQC3GX5lSGSWJBAAh1fdnSHUjjdOBmXqIvcCig==", 2023-12-16T21:43:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.439 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-16T21:43:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.x", 2023-12-16T21:43:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCSGX5lR1isJhAAfuAWAQcooIEu0EhCOvyIpQ==", 2023-12-16T21:43:14.440 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.441 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-16T21:43:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-12-16T21:43:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.442 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "entity": "mgr.y", 2023-12-16T21:43:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "key": "AQCSGX5lPwTqIxAAvyqGdtQlV+u1hf32PLcwzA==", 2023-12-16T21:43:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "caps": { 2023-12-16T21:43:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "mds": "allow *", 2023-12-16T21:43:14.443 INFO:teuthology.orchestra.run.smithi046.stdout: "mon": "allow profile mgr", 2023-12-16T21:43:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: "osd": "allow *" 2023-12-16T21:43:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:14.444 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2023-12-16T21:43:14.444 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f01500664c0 msgr2=0x7f0150068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f01500664c0 0x7f0150068980 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f016409de40 tx=0x7f0158008040 comp rx=0 tx=0).stop 2023-12-16T21:43:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 msgr2=0x7f016413ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f016413ab10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f0154007040 tx=0x7f014c005800 comp rx=0 tx=0).stop 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 shutdown_connections 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f01500664c0 0x7f0150068980 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f016413cba0 0x7f01641dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f016413b9d0 0x7f016413c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 --2- 172.21.15.46:0/1717687535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f014c0039b0 0x7f016413ab10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 >> 172.21.15.46:0/1717687535 conn(0x7f01640b48e0 msgr2=0x7f01640b5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 shutdown_connections 2023-12-16T21:43:14.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.411+0000 7f0169dda640 1 -- 172.21.15.46:0/1717687535 wait complete. 2023-12-16T21:43:14.449 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-16T21:43:14.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.559+0000 7f7ead551640 1 Processor -- start 2023-12-16T21:43:14.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.559+0000 7f7ead551640 1 -- start start 2023-12-16T21:43:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ea80ba4f0 0x7f7ea813e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ea80b7b60 0x7f7ea813e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea80ba8d0 con 0x7f7ea80b9500 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80b1500 con 0x7f7ea80b7b60 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80b7ff0 con 0x7f7ea80ba4f0 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ea80b7b60 0x7f7ea813e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f7ea80b7b60 0x7f7ea813e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52004/0 (socket says 172.21.15.46:52004) 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 -- 172.21.15.46:0/3589058541 learned_addr learned my addr 172.21.15.46:0/3589058541 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ea6ffd640 1 --2- 172.21.15.46:0/3589058541 >> v2:172.21.15.161:3300/0 conn(0x7f7ea80ba4f0 0x7f7ea813e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 -- 172.21.15.46:0/3589058541 >> v2:172.21.15.161:3300/0 conn(0x7f7ea80ba4f0 msgr2=0x7f7ea813e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 --2- 172.21.15.46:0/3589058541 >> v2:172.21.15.161:3300/0 conn(0x7f7ea80ba4f0 0x7f7ea813e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 -- 172.21.15.46:0/3589058541 >> v1:172.21.15.161:6789/0 conn(0x7f7ea80b9500 legacy=0x7f7ea8147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 -- 172.21.15.46:0/3589058541 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80b82d0 con 0x7f7ea80b7b60 2023-12-16T21:43:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ea6ffd640 1 --2- 172.21.15.46:0/3589058541 >> v2:172.21.15.161:3300/0 conn(0x7f7ea80ba4f0 0x7f7ea813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7e9ffff640 1 --2- 172.21.15.46:0/3589058541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b7b60 0x7f7ea813e840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7ea806b4c0 tx=0x7f7e9001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af424b2195c0b87 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ea67fc640 1 -- 172.21.15.46:0/3589058541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e900ab620 con 0x7f7ea80b7b60 2023-12-16T21:43:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ea67fc640 1 -- 172.21.15.46:0/3589058541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e900ab780 con 0x7f7ea80b7b60 2023-12-16T21:43:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ea67fc640 1 -- 172.21.15.46:0/3589058541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e900abb10 con 0x7f7ea80b7b60 2023-12-16T21:43:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- 172.21.15.46:0/3589058541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b7b60 msgr2=0x7f7ea813e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 --2- 172.21.15.46:0/3589058541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b7b60 0x7f7ea813e840 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7ea806b4c0 tx=0x7f7e9001c490 comp rx=0 tx=0).stop 2023-12-16T21:43:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- 172.21.15.46:0/3589058541 shutdown_connections 2023-12-16T21:43:14.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 --2- 172.21.15.46:0/3589058541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b7b60 0x7f7ea813e840 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 --2- 172.21.15.46:0/3589058541 >> v2:172.21.15.161:3300/0 conn(0x7f7ea80ba4f0 0x7f7ea813e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- 172.21.15.46:0/3589058541 >> 172.21.15.46:0/3589058541 conn(0x7f7ea806a980 msgr2=0x7f7ea806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- 172.21.15.46:0/3589058541 shutdown_connections 2023-12-16T21:43:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.563+0000 7f7ead551640 1 -- 172.21.15.46:0/3589058541 wait complete. 2023-12-16T21:43:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 Processor -- start 2023-12-16T21:43:14.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 -- start start 2023-12-16T21:43:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b9500 0x7f7ea81e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea80ba4f0 0x7f7ea8138e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea806d3b0 0x7f7ea81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8142200 con 0x7f7ea80b9500 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8142c50 con 0x7f7ea806d3b0 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea8141f70 con 0x7f7ea80ba4f0 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b9500 0x7f7ea81e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b9500 0x7f7ea81e23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52016/0 (socket says 172.21.15.46:52016) 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 -- 172.21.15.46:0/2109526676 learned_addr learned my addr 172.21.15.46:0/2109526676 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea77fe640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea806d3b0 0x7f7ea81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 -- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea80ba4f0 msgr2=0x7f7ea8138e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7e9ffff640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea80ba4f0 0x7f7ea8138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea80ba4f0 0x7f7ea8138e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 -- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea806d3b0 msgr2=0x7f7ea81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea806d3b0 0x7f7ea81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 -- 172.21.15.46:0/2109526676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e900094d0 con 0x7f7ea80b9500 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea77fe640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea806d3b0 0x7f7ea81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7e9ffff640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea80ba4f0 0x7f7ea8138e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ea6ffd640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b9500 0x7f7ea81e23f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d800 tx=0x7f7e9800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7e9f7fe640 1 -- 172.21.15.46:0/2109526676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800b6c0 con 0x7f7ea80b9500 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7e9f7fe640 1 -- 172.21.15.46:0/2109526676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9800b820 con 0x7f7ea80b9500 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea8142520 con 0x7f7ea80b9500 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7e9f7fe640 1 -- 172.21.15.46:0/2109526676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800bbd0 con 0x7f7ea80b9500 2023-12-16T21:43:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea8139900 con 0x7f7ea80b9500 2023-12-16T21:43:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.567+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea80689d0 con 0x7f7ea80b9500 2023-12-16T21:43:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.571+0000 7f7e9f7fe640 1 -- 172.21.15.46:0/2109526676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7e9800bd30 con 0x7f7ea80b9500 2023-12-16T21:43:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.571+0000 7f7e9f7fe640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7e88066470 0x7f7e88068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.571+0000 7f7e9f7fe640 1 -- 172.21.15.46:0/2109526676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7e980849d0 con 0x7f7ea80b9500 2023-12-16T21:43:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.571+0000 7f7e9ffff640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7e88066470 0x7f7e88068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.571+0000 7f7e9ffff640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7e88066470 0x7f7e88068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e900af000 tx=0x7f7e90008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.575+0000 7f7e9f7fe640 1 -- 172.21.15.46:0/2109526676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7e98052310 con 0x7f7ea80b9500 2023-12-16T21:43:14.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.787+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7ea813c210 con 0x7f7ea80b9500 2023-12-16T21:43:14.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.791+0000 7f7e9f7fe640 1 -- 172.21.15.46:0/2109526676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7e9805ae60 con 0x7f7ea80b9500 2023-12-16T21:43:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7e88066470 msgr2=0x7f7e88068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7e88066470 0x7f7e88068930 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7e900af000 tx=0x7f7e90008040 comp rx=0 tx=0).stop 2023-12-16T21:43:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b9500 msgr2=0x7f7ea81e23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b9500 0x7f7ea81e23f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d800 tx=0x7f7e9800dcc0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 shutdown_connections 2023-12-16T21:43:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7e88066470 0x7f7e88068930 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ea806d3b0 0x7f7ea81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ea80ba4f0 0x7f7ea8138e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 --2- 172.21.15.46:0/2109526676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ea80b9500 0x7f7ea81e23f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 >> 172.21.15.46:0/2109526676 conn(0x7f7ea806a980 msgr2=0x7f7ea813e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 shutdown_connections 2023-12-16T21:43:14.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.795+0000 7f7ead551640 1 -- 172.21.15.46:0/2109526676 wait complete. 2023-12-16T21:43:14.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-16T21:43:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f9679d640 1 Processor -- start 2023-12-16T21:43:14.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f9679d640 1 -- start start 2023-12-16T21:43:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f9679d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900af6e0 0x7f8f9013d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f9679d640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8f9013d700 0x7f8f9014dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f9679d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9013db90 con 0x7f8f9006fa20 2023-12-16T21:43:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f9679d640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f900714d0 con 0x7f8f9013d700 2023-12-16T21:43:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f9679d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900b6430 con 0x7f8f900af6e0 2023-12-16T21:43:14.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f8ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8f9006fa20 0x7f8f90070790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46592/0 (socket says 172.21.15.46:46592) 2023-12-16T21:43:14.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.923+0000 7f8f8ffff640 1 -- 172.21.15.46:0/3189828886 learned_addr learned my addr 172.21.15.46:0/3189828886 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f87fff640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900af6e0 0x7f8f9013d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f94d3a640 1 --2- 172.21.15.46:0/3189828886 >> v2:172.21.15.161:3300/0 conn(0x7f8f9013d700 0x7f8f9014dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3274901221 0 0) 0x7f8f9013db90 con 0x7f8f9006fa20 2023-12-16T21:43:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f7c005740 con 0x7f8f9006fa20 2023-12-16T21:43:14.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3875352457 0 0) 0x7f8f7c005740 con 0x7f8f9006fa20 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900af6e0 msgr2=0x7f8f9013d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900af6e0 0x7f8f9013d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 >> v2:172.21.15.161:3300/0 conn(0x7f8f9013d700 msgr2=0x7f8f9014dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 --2- 172.21.15.46:0/3189828886 >> v2:172.21.15.161:3300/0 conn(0x7f8f9013d700 0x7f8f9014dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900b5530 con 0x7f8f9006fa20 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f8f80001680 con 0x7f8f9006fa20 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 >> v1:172.21.15.46:6789/0 conn(0x7f8f9006fa20 legacy=0x7f8f90070790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f7c005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900af6e0 0x7f8f7c0069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f7c006f20 0x7f8f7c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900714d0 con 0x7f8f7c005b90 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c00dba0 con 0x7f8f7c006f20 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c00dd60 con 0x7f8f900af6e0 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8ffff640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f7c005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8ffff640 1 -- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900af6e0 msgr2=0x7f8f7c0069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8ffff640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900af6e0 0x7f8f7c0069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f94d3a640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f7c006f20 0x7f8f7c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8ffff640 1 -- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f7c006f20 msgr2=0x7f8f7c0073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8ffff640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f7c006f20 0x7f8f7c0073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8ffff640 1 -- 172.21.15.46:0/3189828886 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c005740 con 0x7f8f7c005b90 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8ffff640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f7c005fb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c210 tx=0x7f8f8001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd04c0ab522099a server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800058e0 con 0x7f8f7c005b90 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80005a40 con 0x7f8f7c005b90 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f8f7fe640 1 -- 172.21.15.46:0/3189828886 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80005e10 con 0x7f8f7c005b90 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f9679d640 1 -- 172.21.15.46:0/3189828886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 msgr2=0x7f8f7c005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f9679d640 1 --2- 172.21.15.46:0/3189828886 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f7c005fb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8f8000c210 tx=0x7f8f8001cc60 comp rx=0 tx=0).stop 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f94d3a640 1 -- 172.21.15.46:0/3189828886 reap_dead start 2023-12-16T21:43:14.937 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f9679d640 1 -- 172.21.15.46:0/3189828886 shutdown_connections 2023-12-16T21:43:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f9679d640 1 -- 172.21.15.46:0/3189828886 >> 172.21.15.46:0/3189828886 conn(0x7f8f9006b860 msgr2=0x7f8f9006bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f9679d640 1 -- 172.21.15.46:0/3189828886 shutdown_connections 2023-12-16T21:43:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.927+0000 7f8f9679d640 1 -- 172.21.15.46:0/3189828886 wait complete. 2023-12-16T21:43:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 Processor -- start 2023-12-16T21:43:14.938 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 -- start start 2023-12-16T21:43:14.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f901e2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f900ad520 0x7f8f901d8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900ad9a0 0x7f8f901d8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.939 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f901e2ca0 con 0x7f8f7c005b90 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f900aebe0 con 0x7f8f900ad520 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f901e30c0 con 0x7f8f900ad9a0 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f901e2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f901e2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52036/0 (socket says 172.21.15.46:52036) 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 -- 172.21.15.46:0/2379865732 learned_addr learned my addr 172.21.15.46:0/2379865732 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 -- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900ad9a0 msgr2=0x7f8f901d8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900ad9a0 0x7f8f901d8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 -- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f900ad520 msgr2=0x7f8f901d8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f900ad520 0x7f8f901d8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 -- 172.21.15.46:0/2379865732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900ac980 con 0x7f8f7c005b90 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8ffff640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f901e2210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ab040 tx=0x7f8f7c007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8d7fa640 1 -- 172.21.15.46:0/2379865732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f800047f0 con 0x7f8f7c005b90 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8d7fa640 1 -- 172.21.15.46:0/2379865732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80004950 con 0x7f8f7c005b90 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f900ae900 con 0x7f8f7c005b90 2023-12-16T21:43:14.940 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8d7fa640 1 -- 172.21.15.46:0/2379865732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f80004d00 con 0x7f8f7c005b90 2023-12-16T21:43:14.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f900af280 con 0x7f8f7c005b90 2023-12-16T21:43:14.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f90070e20 con 0x7f8f7c005b90 2023-12-16T21:43:14.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8d7fa640 1 -- 172.21.15.46:0/2379865732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8f80007070 con 0x7f8f7c005b90 2023-12-16T21:43:14.941 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8d7fa640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8f700664a0 0x7f8f70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.931+0000 7f8f8d7fa640 1 -- 172.21.15.46:0/2379865732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f8011d560 con 0x7f8f7c005b90 2023-12-16T21:43:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.935+0000 7f8f87fff640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8f700664a0 0x7f8f70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:14.942 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.935+0000 7f8f87fff640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8f700664a0 0x7f8f70068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f78006aa0 tx=0x7f8f78006a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:14.944 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:14.935+0000 7f8f8d7fa640 1 -- 172.21.15.46:0/2379865732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8f800eadc0 con 0x7f8f7c005b90 2023-12-16T21:43:15.155 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.147+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f8f900689d0 con 0x7f8f7c005b90 2023-12-16T21:43:15.159 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.151+0000 7f8f8d7fa640 1 -- 172.21.15.46:0/2379865732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8f80004e60 con 0x7f8f7c005b90 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8f700664a0 msgr2=0x7f8f70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8f700664a0 0x7f8f70068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f78006aa0 tx=0x7f8f78006a30 comp rx=0 tx=0).stop 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 msgr2=0x7f8f901e2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f901e2210 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8f800ab040 tx=0x7f8f7c007a10 comp rx=0 tx=0).stop 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 shutdown_connections 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8f700664a0 0x7f8f70068960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8f900ad9a0 0x7f8f901d8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8f900ad520 0x7f8f901d8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 --2- 172.21.15.46:0/2379865732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f7c005b90 0x7f8f901e2210 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 >> 172.21.15.46:0/2379865732 conn(0x7f8f9006b860 msgr2=0x7f8f900b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:15.163 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 shutdown_connections 2023-12-16T21:43:15.164 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.155+0000 7f8f9679d640 1 -- 172.21.15.46:0/2379865732 wait complete. 2023-12-16T21:43:15.179 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-16T21:43:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee9b0f640 1 Processor -- start 2023-12-16T21:43:15.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee9b0f640 1 -- start start 2023-12-16T21:43:15.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee9b0f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee406c1a0 0x7fdee40b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee9b0f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdee40b1f80 0x7fdee414d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.292 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee9b0f640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee40af330 con 0x7fdee40adf50 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee9b0f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee40aed20 con 0x7fdee406c1a0 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee9b0f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee406c580 con 0x7fdee40b1f80 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee406c1a0 0x7fdee40b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdee406c1a0 0x7fdee40b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52052/0 (socket says 172.21.15.46:52052) 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee37fe640 1 -- 172.21.15.46:0/523229599 learned_addr learned my addr 172.21.15.46:0/523229599 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee3fff640 1 --2- 172.21.15.46:0/523229599 >> v2:172.21.15.161:3300/0 conn(0x7fdee40b1f80 0x7fdee414d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee37fe640 1 -- 172.21.15.46:0/523229599 >> v2:172.21.15.161:3300/0 conn(0x7fdee40b1f80 msgr2=0x7fdee414d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee37fe640 1 --2- 172.21.15.46:0/523229599 >> v2:172.21.15.161:3300/0 conn(0x7fdee40b1f80 0x7fdee414d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee37fe640 1 -- 172.21.15.46:0/523229599 >> v1:172.21.15.161:6789/0 conn(0x7fdee40adf50 legacy=0x7fdee40b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:15.293 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.283+0000 7fdee37fe640 1 -- 172.21.15.46:0/523229599 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee40ad600 con 0x7fdee406c1a0 2023-12-16T21:43:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee37fe640 1 --2- 172.21.15.46:0/523229599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee40b2790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdee4072680 tx=0x7fdecc01ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40603889d8fab66 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee2ffd640 1 -- 172.21.15.46:0/523229599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0a7b20 con 0x7fdee406c1a0 2023-12-16T21:43:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee2ffd640 1 -- 172.21.15.46:0/523229599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc0a7c80 con 0x7fdee406c1a0 2023-12-16T21:43:15.295 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee2ffd640 1 -- 172.21.15.46:0/523229599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0af440 con 0x7fdee406c1a0 2023-12-16T21:43:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 -- 172.21.15.46:0/523229599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 msgr2=0x7fdee40b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.296 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/523229599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee40b2790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdee4072680 tx=0x7fdecc01ead0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 -- 172.21.15.46:0/523229599 shutdown_connections 2023-12-16T21:43:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/523229599 >> v2:172.21.15.161:3300/0 conn(0x7fdee40b1f80 0x7fdee414d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/523229599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee40b2790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 -- 172.21.15.46:0/523229599 >> 172.21.15.46:0/523229599 conn(0x7fdee406a580 msgr2=0x7fdee406a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 -- 172.21.15.46:0/523229599 shutdown_connections 2023-12-16T21:43:15.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.287+0000 7fdee9b0f640 1 -- 172.21.15.46:0/523229599 wait complete. 2023-12-16T21:43:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 Processor -- start 2023-12-16T21:43:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 -- start start 2023-12-16T21:43:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee413f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee40adf50 0x7fdee41db020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee40b1f80 0x7fdee41db560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee41dabf0 con 0x7fdee406c1a0 2023-12-16T21:43:15.298 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee406f360 con 0x7fdee40adf50 2023-12-16T21:43:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee413dda0 con 0x7fdee40b1f80 2023-12-16T21:43:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee413f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52060/0 (socket says 172.21.15.46:52060) 2023-12-16T21:43:15.299 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 -- 172.21.15.46:0/3055256111 learned_addr learned my addr 172.21.15.46:0/3055256111 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee3fff640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee40b1f80 0x7fdee41db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdedbfff640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee40adf50 0x7fdee41db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 -- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee40b1f80 msgr2=0x7fdee41db560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee40b1f80 0x7fdee41db560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 -- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee40adf50 msgr2=0x7fdee41db020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee40adf50 0x7fdee41db020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 -- 172.21.15.46:0/3055256111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdecc00b770 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee37fe640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee413f260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00b2c0 tx=0x7fdecc01b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee0ff9640 1 -- 172.21.15.46:0/3055256111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc006ee0 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee0ff9640 1 -- 172.21.15.46:0/3055256111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc0afca0 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee0ff9640 1 -- 172.21.15.46:0/3055256111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0a7b20 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee413e040 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee41436b0 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee41438a0 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.291+0000 7fdee0ff9640 1 -- 172.21.15.46:0/3055256111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fdecc0b6050 con 0x7fdee406c1a0 2023-12-16T21:43:15.303 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.295+0000 7fdee0ff9640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdec40664a0 0x7fdec4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.295+0000 7fdee0ff9640 1 -- 172.21.15.46:0/3055256111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdecc120d20 con 0x7fdee406c1a0 2023-12-16T21:43:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.295+0000 7fdedbfff640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdec40664a0 0x7fdec4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.295+0000 7fdedbfff640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdec40664a0 0x7fdec4068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdee413ebf0 tx=0x7fded4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:15.304 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.299+0000 7fdee0ff9640 1 -- 172.21.15.46:0/3055256111 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fdecc0ee620 con 0x7fdee406c1a0 2023-12-16T21:43:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.507+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fdee40679e0 con 0x7fdee406c1a0 2023-12-16T21:43:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee0ff9640 1 -- 172.21.15.46:0/3055256111 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdecc0b4020 con 0x7fdee406c1a0 2023-12-16T21:43:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdec40664a0 msgr2=0x7fdec4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdec40664a0 0x7fdec4068960 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdee413ebf0 tx=0x7fded4009040 comp rx=0 tx=0).stop 2023-12-16T21:43:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 msgr2=0x7fdee413f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee413f260 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdecc00b2c0 tx=0x7fdecc01b870 comp rx=0 tx=0).stop 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 shutdown_connections 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdec40664a0 0x7fdec4068960 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdee40b1f80 0x7fdee41db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdee40adf50 0x7fdee41db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 --2- 172.21.15.46:0/3055256111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdee406c1a0 0x7fdee413f260 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 >> 172.21.15.46:0/3055256111 conn(0x7fdee406a580 msgr2=0x7fdee413c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 shutdown_connections 2023-12-16T21:43:15.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.511+0000 7fdee9b0f640 1 -- 172.21.15.46:0/3055256111 wait complete. 2023-12-16T21:43:15.536 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-16T21:43:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f808d5d5640 1 Processor -- start 2023-12-16T21:43:15.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f808d5d5640 1 -- start start 2023-12-16T21:43:15.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f808d5d5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80880b3b00 0x7f80880ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f808d5d5640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f808806b6b0 0x7f808806bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f808d5d5640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80880af850 con 0x7f808806c070 2023-12-16T21:43:15.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f808d5d5640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80880ad7f0 con 0x7f808806b6b0 2023-12-16T21:43:15.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f808d5d5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808806c4e0 con 0x7f80880b3b00 2023-12-16T21:43:15.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f8086ffd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80880b3b00 0x7f80880ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f80867fc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f808806b6b0 0x7f808806bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f8086ffd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80880b3b00 0x7f80880ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:39022/0 (socket says 172.21.15.46:39022) 2023-12-16T21:43:15.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f8086ffd640 1 -- 172.21.15.46:0/2778575101 learned_addr learned my addr 172.21.15.46:0/2778575101 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:15.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f80867fc640 1 -- 172.21.15.46:0/2778575101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80880b3b00 msgr2=0x7f80880ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f80867fc640 1 --2- 172.21.15.46:0/2778575101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80880b3b00 0x7f80880ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f80867fc640 1 -- 172.21.15.46:0/2778575101 >> v1:172.21.15.161:6789/0 conn(0x7f808806c070 legacy=0x7f8088146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.649 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.639+0000 7f80867fc640 1 -- 172.21.15.46:0/2778575101 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80880b3ee0 con 0x7f808806b6b0 2023-12-16T21:43:15.650 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f80867fc640 1 --2- 172.21.15.46:0/2778575101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808806b6b0 0x7f808806bb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80880b40d0 tx=0x7f807001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6129016011b9fe6 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f8085ffb640 1 -- 172.21.15.46:0/2778575101 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700ac600 con 0x7f808806b6b0 2023-12-16T21:43:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f8085ffb640 1 -- 172.21.15.46:0/2778575101 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80700a8050 con 0x7f808806b6b0 2023-12-16T21:43:15.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f8085ffb640 1 -- 172.21.15.46:0/2778575101 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700ac980 con 0x7f808806b6b0 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 -- 172.21.15.46:0/2778575101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808806b6b0 msgr2=0x7f808806bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 --2- 172.21.15.46:0/2778575101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808806b6b0 0x7f808806bb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80880b40d0 tx=0x7f807001c480 comp rx=0 tx=0).stop 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 -- 172.21.15.46:0/2778575101 shutdown_connections 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 --2- 172.21.15.46:0/2778575101 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f808806b6b0 0x7f808806bb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 --2- 172.21.15.46:0/2778575101 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f80880b3b00 0x7f80880ae070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 -- 172.21.15.46:0/2778575101 >> 172.21.15.46:0/2778575101 conn(0x7f808806a580 msgr2=0x7f808806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 -- 172.21.15.46:0/2778575101 shutdown_connections 2023-12-16T21:43:15.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.643+0000 7f808d5d5640 1 -- 172.21.15.46:0/2778575101 wait complete. 2023-12-16T21:43:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 Processor -- start 2023-12-16T21:43:15.653 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 -- start start 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f808806b6b0 0x7f808809ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808806c070 0x7f808809b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80880b3b00 0x7f8088071100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80880717a0 con 0x7f808806c070 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808813acb0 con 0x7f80880b3b00 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80880702d0 con 0x7f808806b6b0 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808806c070 0x7f808809b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808806c070 0x7f808809b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52070/0 (socket says 172.21.15.46:52070) 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 -- 172.21.15.46:0/787993631 learned_addr learned my addr 172.21.15.46:0/787993631 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f8086ffd640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f808806b6b0 0x7f808809ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80877fe640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80880b3b00 0x7f8088071100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 -- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f808806b6b0 msgr2=0x7f808809ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f808806b6b0 0x7f808809ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 -- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80880b3b00 msgr2=0x7f8088071100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80880b3b00 0x7f8088071100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 -- 172.21.15.46:0/787993631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8070009420 con 0x7f808806c070 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80877fe640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80880b3b00 0x7f8088071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f80867fc640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808806c070 0x7f808809b340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c10 tx=0x7f8070019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f807f7fe640 1 -- 172.21.15.46:0/787993631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700b5410 con 0x7f808806c070 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f807f7fe640 1 -- 172.21.15.46:0/787993631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80700b5590 con 0x7f808806c070 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f807f7fe640 1 -- 172.21.15.46:0/787993631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80700b5940 con 0x7f808806c070 2023-12-16T21:43:15.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80880b2270 con 0x7f808806c070 2023-12-16T21:43:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808809c680 con 0x7f808806c070 2023-12-16T21:43:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.647+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80880679e0 con 0x7f808806c070 2023-12-16T21:43:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.651+0000 7f807f7fe640 1 -- 172.21.15.46:0/787993631 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f80700b3040 con 0x7f808806c070 2023-12-16T21:43:15.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.651+0000 7f807f7fe640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8068066470 0x7f8068068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.651+0000 7f807f7fe640 1 -- 172.21.15.46:0/787993631 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f807011d9d0 con 0x7f808806c070 2023-12-16T21:43:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.651+0000 7f8086ffd640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8068066470 0x7f8068068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:15.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.651+0000 7f8086ffd640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8068066470 0x7f8068068930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8080011340 tx=0x7f808000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:15.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.655+0000 7f807f7fe640 1 -- 172.21.15.46:0/787993631 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f80700eb2e0 con 0x7f808806c070 2023-12-16T21:43:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.859+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f808809cba0 con 0x7f808806c070 2023-12-16T21:43:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f807f7fe640 1 -- 172.21.15.46:0/787993631 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f808809cba0 con 0x7f808806c070 2023-12-16T21:43:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8068066470 msgr2=0x7f8068068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8068066470 0x7f8068068930 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8080011340 tx=0x7f808000b040 comp rx=0 tx=0).stop 2023-12-16T21:43:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808806c070 msgr2=0x7f808809b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808806c070 0x7f808809b340 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8070000c10 tx=0x7f8070019220 comp rx=0 tx=0).stop 2023-12-16T21:43:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 shutdown_connections 2023-12-16T21:43:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8068066470 0x7f8068068930 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f80880b3b00 0x7f8088071100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f808806c070 0x7f808809b340 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 --2- 172.21.15.46:0/787993631 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f808806b6b0 0x7f808809ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 >> 172.21.15.46:0/787993631 conn(0x7f808806a580 msgr2=0x7f80880b4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 shutdown_connections 2023-12-16T21:43:15.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:15.863+0000 7f808d5d5640 1 -- 172.21.15.46:0/787993631 wait complete. 2023-12-16T21:43:15.887 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-16T21:43:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6fe1f4640 1 Processor -- start 2023-12-16T21:43:16.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6fe1f4640 1 -- start start 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6fe1f4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb6f80af4f0 0x7fb6f809caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6fe1f4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f80722c0 0x7fb6f809cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6fe1f4640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f80b1800 con 0x7fb6f80af8d0 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6fe1f4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f80b1610 con 0x7fb6f80722c0 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6fe1f4640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f80b1aa0 con 0x7fb6f80af4f0 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f80722c0 0x7fb6f809cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f6ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb6f80722c0 0x7fb6f809cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52078/0 (socket says 172.21.15.46:52078) 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f6ffd640 1 -- 172.21.15.46:0/670762525 learned_addr learned my addr 172.21.15.46:0/670762525 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f77fe640 1 --2- 172.21.15.46:0/670762525 >> v2:172.21.15.161:3300/0 conn(0x7fb6f80af4f0 0x7fb6f809caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f6ffd640 1 -- 172.21.15.46:0/670762525 >> v2:172.21.15.161:3300/0 conn(0x7fb6f80af4f0 msgr2=0x7fb6f809caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f6ffd640 1 --2- 172.21.15.46:0/670762525 >> v2:172.21.15.161:3300/0 conn(0x7fb6f80af4f0 0x7fb6f809caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f6ffd640 1 -- 172.21.15.46:0/670762525 >> v1:172.21.15.161:6789/0 conn(0x7fb6f80af8d0 legacy=0x7fb6f8148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.007+0000 7fb6f6ffd640 1 -- 172.21.15.46:0/670762525 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f8147b10 con 0x7fb6f80722c0 2023-12-16T21:43:16.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f6ffd640 1 --2- 172.21.15.46:0/670762525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f809cfe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2848d973ec55e427 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f67fc640 1 -- 172.21.15.46:0/670762525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0ab600 con 0x7fb6f80722c0 2023-12-16T21:43:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f67fc640 1 -- 172.21.15.46:0/670762525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0a7050 con 0x7fb6f80722c0 2023-12-16T21:43:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f67fc640 1 -- 172.21.15.46:0/670762525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0ab980 con 0x7fb6f80722c0 2023-12-16T21:43:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/670762525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 msgr2=0x7fb6f809cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/670762525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f809cfe0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec009870 tx=0x7fb6ec01c430 comp rx=0 tx=0).stop 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/670762525 shutdown_connections 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/670762525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f809cfe0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/670762525 >> v2:172.21.15.161:3300/0 conn(0x7fb6f80af4f0 0x7fb6f809caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/670762525 >> 172.21.15.46:0/670762525 conn(0x7fb6f806a580 msgr2=0x7fb6f806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/670762525 shutdown_connections 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/670762525 wait complete. 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 Processor -- start 2023-12-16T21:43:16.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- start start 2023-12-16T21:43:16.021 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f8146e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f80af4f0 0x7fb6f81df130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f80af8d0 0x7fb6f81df670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81e3280 con 0x7fb6f80722c0 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81e2bc0 con 0x7fb6f80af4f0 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6fe1f4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f81e2ea0 con 0x7fb6f80af8d0 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f8146e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f8146e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52080/0 (socket says 172.21.15.46:52080) 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f77fe640 1 -- 172.21.15.46:0/2187278948 learned_addr learned my addr 172.21.15.46:0/2187278948 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f7fff640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f80af8d0 0x7fb6f81df670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.011+0000 7fb6f6ffd640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f80af4f0 0x7fb6f81df130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f77fe640 1 -- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f80af8d0 msgr2=0x7fb6f81df670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f77fe640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f80af8d0 0x7fb6f81df670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f77fe640 1 -- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f80af4f0 msgr2=0x7fb6f81df130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f77fe640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f80af4f0 0x7fb6f81df130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f77fe640 1 -- 172.21.15.46:0/2187278948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0094d0 con 0x7fb6f80722c0 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f7fff640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f80af8d0 0x7fb6f81df670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f77fe640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f8146e80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d800 tx=0x7fb6e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6dffff640 1 -- 172.21.15.46:0/2187278948 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800b6c0 con 0x7fb6f80722c0 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6dffff640 1 -- 172.21.15.46:0/2187278948 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e800b820 con 0x7fb6f80722c0 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6dffff640 1 -- 172.21.15.46:0/2187278948 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e800bb90 con 0x7fb6f80722c0 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f81e26a0 con 0x7fb6f80722c0 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f809c690 con 0x7fb6f80722c0 2023-12-16T21:43:16.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f80679e0 con 0x7fb6f80722c0 2023-12-16T21:43:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6dffff640 1 -- 172.21.15.46:0/2187278948 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fb6e800bcf0 con 0x7fb6f80722c0 2023-12-16T21:43:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6dffff640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb6e4066470 0x7fb6e4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6dffff640 1 -- 172.21.15.46:0/2187278948 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb6e8084a90 con 0x7fb6f80722c0 2023-12-16T21:43:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f6ffd640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb6e4066470 0x7fb6e4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.015+0000 7fb6f6ffd640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb6e4066470 0x7fb6e4068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80b5c60 tx=0x7fb6ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.019+0000 7fb6dffff640 1 -- 172.21.15.46:0/2187278948 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fb6e80523d0 con 0x7fb6f80722c0 2023-12-16T21:43:16.231 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.223+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb6f80689d0 con 0x7fb6f80722c0 2023-12-16T21:43:16.236 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.227+0000 7fb6dffff640 1 -- 172.21.15.46:0/2187278948 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb6e80559f0 con 0x7fb6f80722c0 2023-12-16T21:43:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb6e4066470 msgr2=0x7fb6e4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb6e4066470 0x7fb6e4068930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb6f80b5c60 tx=0x7fb6ec009040 comp rx=0 tx=0).stop 2023-12-16T21:43:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 msgr2=0x7fb6f8146e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f8146e80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800d800 tx=0x7fb6e800dcc0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 shutdown_connections 2023-12-16T21:43:16.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb6e4066470 0x7fb6e4068930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb6f80af8d0 0x7fb6f81df670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb6f80af4f0 0x7fb6f81df130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 --2- 172.21.15.46:0/2187278948 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb6f80722c0 0x7fb6f8146e80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 >> 172.21.15.46:0/2187278948 conn(0x7fb6f806a580 msgr2=0x7fb6f80b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 shutdown_connections 2023-12-16T21:43:16.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.231+0000 7fb6fe1f4640 1 -- 172.21.15.46:0/2187278948 wait complete. 2023-12-16T21:43:16.253 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-16T21:43:16.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14af5fe640 1 Processor -- start 2023-12-16T21:43:16.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14af5fe640 1 -- start start 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14af5fe640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f14a80afb30 0x7f14a806fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14af5fe640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a80b60c0 con 0x7f14a80b5ce0 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14af5fe640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a8071600 con 0x7f14a80720c0 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14af5fe640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a80703f0 con 0x7f14a80afb30 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14ad39a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f14a80b5ce0 0x7f14a8149be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46608/0 (socket says 172.21.15.46:46608) 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14ad39a640 1 -- 172.21.15.46:0/79359053 learned_addr learned my addr 172.21.15.46:0/79359053 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062489680 0 0) 0x7f14a80b60c0 con 0x7f14a80b5ce0 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1494003530 con 0x7f14a80b5ce0 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14adb9b640 1 --2- 172.21.15.46:0/79359053 >> v2:172.21.15.161:3300/0 conn(0x7f14a80afb30 0x7f14a806fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.377 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3797630694 0 0) 0x7f1494003530 con 0x7f14a80b5ce0 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 >> v2:172.21.15.161:3300/0 conn(0x7f14a80afb30 msgr2=0x7f14a806fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 --2- 172.21.15.46:0/79359053 >> v2:172.21.15.161:3300/0 conn(0x7f14a80afb30 0x7f14a806fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 >> v1:172.21.15.161:6789/0 conn(0x7f14a80720c0 legacy=0x7f14a81404c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a80718c0 con 0x7f14a80b5ce0 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f14a4001680 con 0x7f14a80b5ce0 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 >> v1:172.21.15.46:6789/0 conn(0x7f14a80b5ce0 legacy=0x7f14a8149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1494003980 0x7f1494003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f1494004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14940052b0 0x7f149400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80703f0 con 0x7f14940047d0 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149400c160 con 0x7f1494003980 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149400c320 con 0x7f14940052b0 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.367+0000 7f14acb99640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f1494004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14ad39a640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1494003980 0x7f1494003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14acb99640 1 -- 172.21.15.46:0/79359053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14940052b0 msgr2=0x7f149400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14acb99640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14940052b0 0x7f149400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14acb99640 1 -- 172.21.15.46:0/79359053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1494003980 msgr2=0x7f1494003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14acb99640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1494003980 0x7f1494003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14acb99640 1 -- 172.21.15.46:0/79359053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1494003530 con 0x7f14940047d0 2023-12-16T21:43:16.379 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14adb9b640 1 -- 172.21.15.46:0/79359053 reap_dead start 2023-12-16T21:43:16.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14acb99640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f1494004c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1498007fd0 tx=0x7f149801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42ddb936ca06716 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980ab600 con 0x7f14940047d0 2023-12-16T21:43:16.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14980a7050 con 0x7f14940047d0 2023-12-16T21:43:16.380 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f149ffff640 1 -- 172.21.15.46:0/79359053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14980ab9c0 con 0x7f14940047d0 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 -- 172.21.15.46:0/79359053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 msgr2=0x7f1494004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f1494004c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1498007fd0 tx=0x7f149801c230 comp rx=0 tx=0).stop 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 -- 172.21.15.46:0/79359053 shutdown_connections 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 --2- 172.21.15.46:0/79359053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f1494004c50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 -- 172.21.15.46:0/79359053 >> 172.21.15.46:0/79359053 conn(0x7f14a806b860 msgr2=0x7f14a806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 -- 172.21.15.46:0/79359053 shutdown_connections 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 -- 172.21.15.46:0/79359053 wait complete. 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 Processor -- start 2023-12-16T21:43:16.381 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.371+0000 7f14af5fe640 1 -- start start 2023-12-16T21:43:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f14a81e2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14a80b09a0 0x7f14a80b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8138ab0 0x7f14a8138f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.383 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a81e38e0 con 0x7f14940047d0 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a81e32e0 con 0x7f14a8138ab0 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a80b2480 con 0x7f14a80b09a0 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f14a81e2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f14a81e2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52094/0 (socket says 172.21.15.46:52094) 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 -- 172.21.15.46:0/2791615093 learned_addr learned my addr 172.21.15.46:0/2791615093 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14adb9b640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8138ab0 0x7f14a8138f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14acb99640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14a80b09a0 0x7f14a80b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 -- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14a80b09a0 msgr2=0x7f14a80b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14a80b09a0 0x7f14a80b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 -- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8138ab0 msgr2=0x7f14a8138f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8138ab0 0x7f14a8138f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 -- 172.21.15.46:0/2791615093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a81e3520 con 0x7f14940047d0 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14adb9b640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8138ab0 0x7f14a8138f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:16.384 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14ad39a640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f14a81e2da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e880 tx=0x7f14a400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f149dffb640 1 -- 172.21.15.46:0/2791615093 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400c870 con 0x7f14940047d0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f149dffb640 1 -- 172.21.15.46:0/2791615093 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400c9f0 con 0x7f14940047d0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a80b1340 con 0x7f14940047d0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f149dffb640 1 -- 172.21.15.46:0/2791615093 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400cda0 con 0x7f14940047d0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a80b26d0 con 0x7f14940047d0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a80679e0 con 0x7f14940047d0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f149dffb640 1 -- 172.21.15.46:0/2791615093 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f14a401d460 con 0x7f14940047d0 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f149dffb640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f148c066470 0x7f148c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.385 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f149dffb640 1 -- 172.21.15.46:0/2791615093 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f14a4083c00 con 0x7f14940047d0 2023-12-16T21:43:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.375+0000 7f14acb99640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f148c066470 0x7f148c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.386 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.379+0000 7f14acb99640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f148c066470 0x7f148c068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14980ad000 tx=0x7f1498003770 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.387 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.379+0000 7f149dffb640 1 -- 172.21.15.46:0/2791615093 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f14a40513c0 con 0x7f14940047d0 2023-12-16T21:43:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.579+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f14a80689d0 con 0x7f14940047d0 2023-12-16T21:43:16.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.583+0000 7f149dffb640 1 -- 172.21.15.46:0/2791615093 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f14a40549e0 con 0x7f14940047d0 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f148c066470 msgr2=0x7f148c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f148c066470 0x7f148c068930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f14980ad000 tx=0x7f1498003770 comp rx=0 tx=0).stop 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 msgr2=0x7f14a81e2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f14a81e2da0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e880 tx=0x7f14a400ed40 comp rx=0 tx=0).stop 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 shutdown_connections 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f148c066470 0x7f148c068930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f14a8138ab0 0x7f14a8138f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f14a80b09a0 0x7f14a80b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 --2- 172.21.15.46:0/2791615093 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f14940047d0 0x7f14a81e2da0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 >> 172.21.15.46:0/2791615093 conn(0x7f14a806b860 msgr2=0x7f14a813f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:16.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 shutdown_connections 2023-12-16T21:43:16.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.587+0000 7f14af5fe640 1 -- 172.21.15.46:0/2791615093 wait complete. 2023-12-16T21:43:16.610 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bfd29640 1 Processor -- start 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bfd29640 1 -- start start 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bfd29640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b806c920 0x7f23b80afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bfd29640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f23b809dd50 0x7f23b813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bfd29640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b80af2e0 con 0x7f23b80b3fd0 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bfd29640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b80b1f50 con 0x7f23b809dd50 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bfd29640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b806cd00 con 0x7f23b806c920 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bd2c4640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f23b80b3fd0 0x7f23b813e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46614/0 (socket says 172.21.15.46:46614) 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bd2c4640 1 -- 172.21.15.46:0/738639158 learned_addr learned my addr 172.21.15.46:0/738639158 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.723+0000 7f23bdac5640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b806c920 0x7f23b80afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 --2- 172.21.15.46:0/738639158 >> v2:172.21.15.161:3300/0 conn(0x7f23b809dd50 0x7f23b813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.736 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4009128382 0 0) 0x7f23b80af2e0 con 0x7f23b80b3fd0 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23a0003530 con 0x7f23b80b3fd0 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2906832165 0 0) 0x7f23a0003530 con 0x7f23b80b3fd0 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b806c920 msgr2=0x7f23b80afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b806c920 0x7f23b80afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 >> v2:172.21.15.161:3300/0 conn(0x7f23b809dd50 msgr2=0x7f23b813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 --2- 172.21.15.46:0/738639158 >> v2:172.21.15.161:3300/0 conn(0x7f23b809dd50 0x7f23b813ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80af530 con 0x7f23b80b3fd0 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f23ac001680 con 0x7f23b80b3fd0 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 >> v1:172.21.15.46:6789/0 conn(0x7f23b80b3fd0 legacy=0x7f23b813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23a0003980 0x7f23a0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b806c920 0x7f23a00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0004d10 0x7f23a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b80b1f50 con 0x7f23a0004d10 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b806cd00 con 0x7f23a0003980 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a000b990 con 0x7f23b806c920 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0004d10 0x7f23a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bdac5640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23a0003980 0x7f23a0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 -- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b806c920 msgr2=0x7f23a00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23b806c920 0x7f23a00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 -- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23a0003980 msgr2=0x7f23a0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23a0003980 0x7f23a0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 -- 172.21.15.46:0/738639158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a0003530 con 0x7f23a0004d10 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0004d10 0x7f23a0005190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23b809e160 tx=0x7f23a801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22eb8b30205072fe server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80a5620 con 0x7f23a0004d10 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a80a5780 con 0x7f23a0004d10 2023-12-16T21:43:16.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bcac3640 1 -- 172.21.15.46:0/738639158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80a5b30 con 0x7f23a0004d10 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bfd29640 1 -- 172.21.15.46:0/738639158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0004d10 msgr2=0x7f23a0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bfd29640 1 --2- 172.21.15.46:0/738639158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23a0004d10 0x7f23a0005190 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f23b809e160 tx=0x7f23a801ca30 comp rx=0 tx=0).stop 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23be2c6640 1 -- 172.21.15.46:0/738639158 reap_dead start 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bfd29640 1 -- 172.21.15.46:0/738639158 shutdown_connections 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bfd29640 1 -- 172.21.15.46:0/738639158 >> 172.21.15.46:0/738639158 conn(0x7f23b80b48e0 msgr2=0x7f23b80b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bfd29640 1 -- 172.21.15.46:0/738639158 shutdown_connections 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.727+0000 7f23bfd29640 1 -- 172.21.15.46:0/738639158 wait complete. 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 Processor -- start 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 -- start start 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23a0004d10 0x7f23b81dc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b809d1d0 0x7f23b809d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.742 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b8140820 0x7f23b8140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81423d0 con 0x7f23b809d1d0 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b81dbee0 con 0x7f23b8140820 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b809db90 con 0x7f23a0004d10 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b809d1d0 0x7f23b809d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b809d1d0 0x7f23b809d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52108/0 (socket says 172.21.15.46:52108) 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 -- 172.21.15.46:0/452802801 learned_addr learned my addr 172.21.15.46:0/452802801 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23be2c6640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b8140820 0x7f23b8140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bdac5640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23a0004d10 0x7f23b81dc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 -- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23a0004d10 msgr2=0x7f23b81dc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.743 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23a0004d10 0x7f23b81dc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 -- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b8140820 msgr2=0x7f23b8140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b8140820 0x7f23b8140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 -- 172.21.15.46:0/452802801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b80b4750 con 0x7f23b809d1d0 2023-12-16T21:43:16.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23be2c6640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b8140820 0x7f23b8140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:16.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bd2c4640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b809d1d0 0x7f23b809d650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00f3c0 tx=0x7f23ac00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.744 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23b67fc640 1 -- 172.21.15.46:0/452802801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac007d00 con 0x7f23b809d1d0 2023-12-16T21:43:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23b67fc640 1 -- 172.21.15.46:0/452802801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac007e80 con 0x7f23b809d1d0 2023-12-16T21:43:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b8141210 con 0x7f23b809d1d0 2023-12-16T21:43:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23b67fc640 1 -- 172.21.15.46:0/452802801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac00a310 con 0x7f23b809d1d0 2023-12-16T21:43:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b8141f70 con 0x7f23b809d1d0 2023-12-16T21:43:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b80679e0 con 0x7f23b809d1d0 2023-12-16T21:43:16.745 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.731+0000 7f23b67fc640 1 -- 172.21.15.46:0/452802801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f23ac00a470 con 0x7f23b809d1d0 2023-12-16T21:43:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.735+0000 7f23b67fc640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f23a4066470 0x7f23a4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.735+0000 7f23b67fc640 1 -- 172.21.15.46:0/452802801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23ac0868a0 con 0x7f23b809d1d0 2023-12-16T21:43:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.735+0000 7f23bdac5640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f23a4066470 0x7f23a4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.735+0000 7f23bdac5640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f23a4066470 0x7f23a4068930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f23b80695a0 tx=0x7f23b000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:16.746 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.739+0000 7f23b67fc640 1 -- 172.21.15.46:0/452802801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f23ac057b90 con 0x7f23b809d1d0 2023-12-16T21:43:16.902 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.895+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23b81db9d0 con 0x7f23b809d1d0 2023-12-16T21:43:16.903 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.895+0000 7f23b67fc640 1 -- 172.21.15.46:0/452802801 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+12857 (secure 0 0 0) 0x7f23b81db9d0 con 0x7f23b809d1d0 2023-12-16T21:43:16.903 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:16.904 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":26,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:14.034910+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f23a4066470 msgr2=0x7f23a4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f23a4066470 0x7f23a4068930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f23b80695a0 tx=0x7f23b000b040 comp rx=0 tx=0).stop 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b809d1d0 msgr2=0x7f23b809d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b809d1d0 0x7f23b809d650 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f23ac00f3c0 tx=0x7f23ac00f880 comp rx=0 tx=0).stop 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 shutdown_connections 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f23a4066470 0x7f23a4068930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f23b8140820 0x7f23b8140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23b809d1d0 0x7f23b809d650 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 --2- 172.21.15.46:0/452802801 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f23a0004d10 0x7f23b81dc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 >> 172.21.15.46:0/452802801 conn(0x7f23b80b48e0 msgr2=0x7f23b806b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 shutdown_connections 2023-12-16T21:43:16.906 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:16.899+0000 7f23bfd29640 1 -- 172.21.15.46:0/452802801 wait complete. 2023-12-16T21:43:16.923 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:43:16.923 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:43:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af9927640 1 Processor -- start 2023-12-16T21:43:17.053 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af9927640 1 -- start start 2023-12-16T21:43:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af9927640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2af40b1eb0 0x7f2af40aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.054 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af9927640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af41451c0 0x7f2af414ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af9927640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40b1730 con 0x7f2af40af300 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af9927640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af406cd70 con 0x7f2af40b1eb0 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af9927640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40b0ad0 con 0x7f2af41451c0 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af2ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2af40b1eb0 0x7f2af40aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2aebfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2af40af300 0x7f2af4144ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46626/0 (socket says 172.21.15.46:46626) 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2aebfff640 1 -- 172.21.15.46:0/2794451771 learned_addr learned my addr 172.21.15.46:0/2794451771 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af37fe640 1 --2- 172.21.15.46:0/2794451771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af41451c0 0x7f2af414ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af2ffd640 1 -- 172.21.15.46:0/2794451771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af41451c0 msgr2=0x7f2af414ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af2ffd640 1 --2- 172.21.15.46:0/2794451771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af41451c0 0x7f2af414ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af2ffd640 1 -- 172.21.15.46:0/2794451771 >> v1:172.21.15.46:6789/0 conn(0x7f2af40af300 legacy=0x7f2af4144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.055 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.047+0000 7f2af2ffd640 1 -- 172.21.15.46:0/2794451771 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af406c7f0 con 0x7f2af40b1eb0 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 --2- 172.21.15.46:0/2794451771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40b1eb0 0x7f2af40aff30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40097c0 tx=0x7f2ae401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=226475b7d947941e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af27fc640 1 -- 172.21.15.46:0/2794451771 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40ad660 con 0x7f2af40b1eb0 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af27fc640 1 -- 172.21.15.46:0/2794451771 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae40ad7c0 con 0x7f2af40b1eb0 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af27fc640 1 -- 172.21.15.46:0/2794451771 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40adb50 con 0x7f2af40b1eb0 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- 172.21.15.46:0/2794451771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40b1eb0 msgr2=0x7f2af40aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 --2- 172.21.15.46:0/2794451771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40b1eb0 0x7f2af40aff30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40097c0 tx=0x7f2ae401c360 comp rx=0 tx=0).stop 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- 172.21.15.46:0/2794451771 shutdown_connections 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 --2- 172.21.15.46:0/2794451771 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af41451c0 0x7f2af414ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 --2- 172.21.15.46:0/2794451771 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40b1eb0 0x7f2af40aff30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- 172.21.15.46:0/2794451771 >> 172.21.15.46:0/2794451771 conn(0x7f2af40b4900 msgr2=0x7f2af40b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- 172.21.15.46:0/2794451771 shutdown_connections 2023-12-16T21:43:17.058 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- 172.21.15.46:0/2794451771 wait complete. 2023-12-16T21:43:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 Processor -- start 2023-12-16T21:43:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- start start 2023-12-16T21:43:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40af300 0x7f2af4070970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.060 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af40b1eb0 0x7f2af413f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41451c0 0x7f2af406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40704e0 con 0x7f2af40af300 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4070180 con 0x7f2af41451c0 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af413f030 con 0x7f2af40b1eb0 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40af300 0x7f2af4070970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40af300 0x7f2af4070970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52116/0 (socket says 172.21.15.46:52116) 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 -- 172.21.15.46:0/2639006126 learned_addr learned my addr 172.21.15.46:0/2639006126 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2aebfff640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af40b1eb0 0x7f2af413f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 -- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af40b1eb0 msgr2=0x7f2af413f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af40b1eb0 0x7f2af413f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 -- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41451c0 msgr2=0x7f2af406fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af37fe640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41451c0 0x7f2af406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41451c0 0x7f2af406fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 -- 172.21.15.46:0/2639006126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae4009420 con 0x7f2af40af300 2023-12-16T21:43:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af2ffd640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40af300 0x7f2af4070970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40b0040 tx=0x7f2ae4019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af0ff9640 1 -- 172.21.15.46:0/2639006126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40b6820 con 0x7f2af40af300 2023-12-16T21:43:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af0ff9640 1 -- 172.21.15.46:0/2639006126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae40b69a0 con 0x7f2af40af300 2023-12-16T21:43:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af413f310 con 0x7f2af40af300 2023-12-16T21:43:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af0ff9640 1 -- 172.21.15.46:0/2639006126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae40b6d50 con 0x7f2af40af300 2023-12-16T21:43:17.062 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.051+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af40710b0 con 0x7f2af40af300 2023-12-16T21:43:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.055+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af4068960 con 0x7f2af40af300 2023-12-16T21:43:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.055+0000 7f2af0ff9640 1 -- 172.21.15.46:0/2639006126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2ae40b4040 con 0x7f2af40af300 2023-12-16T21:43:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.055+0000 7f2af0ff9640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2ad40664a0 0x7f2ad4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.055+0000 7f2af0ff9640 1 -- 172.21.15.46:0/2639006126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2ae411e4c0 con 0x7f2af40af300 2023-12-16T21:43:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.055+0000 7f2aebfff640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2ad40664a0 0x7f2ad4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.063 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.055+0000 7f2aebfff640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2ad40664a0 0x7f2ad4068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2af4141f90 tx=0x7f2adc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.059+0000 7f2af0ff9640 1 -- 172.21.15.46:0/2639006126 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2ae40ebe40 con 0x7f2af40af300 2023-12-16T21:43:17.220 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.211+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2af40b5380 con 0x7f2af40af300 2023-12-16T21:43:17.221 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af0ff9640 1 -- 172.21.15.46:0/2639006126 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2ae40ef460 con 0x7f2af40af300 2023-12-16T21:43:17.221 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2ad40664a0 msgr2=0x7f2ad4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2ad40664a0 0x7f2ad4068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2af4141f90 tx=0x7f2adc009040 comp rx=0 tx=0).stop 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40af300 msgr2=0x7f2af4070970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40af300 0x7f2af4070970 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40b0040 tx=0x7f2ae4019100 comp rx=0 tx=0).stop 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 shutdown_connections 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2ad40664a0 0x7f2ad4068960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2af41451c0 0x7f2af406fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2af40b1eb0 0x7f2af413f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.224 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 --2- 172.21.15.46:0/2639006126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2af40af300 0x7f2af4070970 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 >> 172.21.15.46:0/2639006126 conn(0x7f2af40b4900 msgr2=0x7f2af4142aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:17.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 shutdown_connections 2023-12-16T21:43:17.225 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.215+0000 7f2af9927640 1 -- 172.21.15.46:0/2639006126 wait complete. 2023-12-16T21:43:17.240 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.355+0000 7f054e413640 1 Processor -- start 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054e413640 1 -- start start 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054e413640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f05480b2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054e413640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f054814e0b0 0x7f0548155440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054e413640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05480b2920 con 0x7f05480bc9b0 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054e413640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480b5df0 con 0x7f054814e0b0 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054e413640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480b1600 con 0x7f05480bc5d0 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f054814e0b0 0x7f0548155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.366 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f054814e0b0 0x7f0548155440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52130/0 (socket says 172.21.15.46:52130) 2023-12-16T21:43:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 -- 172.21.15.46:0/2011815868 learned_addr learned my addr 172.21.15.46:0/2011815868 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f0547fff640 1 --2- 172.21.15.46:0/2011815868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f05480b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 -- 172.21.15.46:0/2011815868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 msgr2=0x7f05480b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 --2- 172.21.15.46:0/2011815868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f05480b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 -- 172.21.15.46:0/2011815868 >> v1:172.21.15.161:6789/0 conn(0x7f05480bc9b0 legacy=0x7f05480b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.367 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 -- 172.21.15.46:0/2011815868 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480b18e0 con 0x7f054814e0b0 2023-12-16T21:43:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f054c9b0640 1 --2- 172.21.15.46:0/2011815868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f054814e0b0 0x7f0548155440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0540009870 tx=0x7f054001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6190e2add7e1faf9 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f05477fe640 1 -- 172.21.15.46:0/2011815868 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400ab640 con 0x7f054814e0b0 2023-12-16T21:43:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f05477fe640 1 -- 172.21.15.46:0/2011815868 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05400a7070 con 0x7f054814e0b0 2023-12-16T21:43:17.368 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.359+0000 7f05477fe640 1 -- 172.21.15.46:0/2011815868 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400ab9c0 con 0x7f054814e0b0 2023-12-16T21:43:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/2011815868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f054814e0b0 msgr2=0x7f0548155440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 --2- 172.21.15.46:0/2011815868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f054814e0b0 0x7f0548155440 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0540009870 tx=0x7f054001c420 comp rx=0 tx=0).stop 2023-12-16T21:43:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/2011815868 shutdown_connections 2023-12-16T21:43:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 --2- 172.21.15.46:0/2011815868 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f054814e0b0 0x7f0548155440 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 --2- 172.21.15.46:0/2011815868 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f05480b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/2011815868 >> 172.21.15.46:0/2011815868 conn(0x7f0548072340 msgr2=0x7f05480bbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:17.369 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/2011815868 shutdown_connections 2023-12-16T21:43:17.370 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/2011815868 wait complete. 2023-12-16T21:43:17.371 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 Processor -- start 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- start start 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f054813e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05480bc9b0 0x7f054814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f054814e0b0 0x7f054814c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054814c380 con 0x7f05480bc9b0 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054813db60 con 0x7f054814e0b0 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05481429c0 con 0x7f05480bc5d0 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05480bc9b0 0x7f054814d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05480bc9b0 0x7f054814d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52138/0 (socket says 172.21.15.46:52138) 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 -- 172.21.15.46:0/1199897750 learned_addr learned my addr 172.21.15.46:0/1199897750 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f0547fff640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f054813e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054c9b0640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f054814e0b0 0x7f054814c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.372 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 -- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 msgr2=0x7f054813e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f054813e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 -- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f054814e0b0 msgr2=0x7f054814c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f054814e0b0 0x7f054814c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 -- 172.21.15.46:0/1199897750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05400094d0 con 0x7f05480bc9b0 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f053f7fe640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05480bc9b0 0x7f054814d710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f053000c4d0 tx=0x7f053000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f05457fa640 1 -- 172.21.15.46:0/1199897750 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05300052d0 con 0x7f05480bc9b0 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f05457fa640 1 -- 172.21.15.46:0/1199897750 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0530005430 con 0x7f05480bc9b0 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0548142ca0 con 0x7f05480bc9b0 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f05457fa640 1 -- 172.21.15.46:0/1199897750 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530012830 con 0x7f05480bc9b0 2023-12-16T21:43:17.373 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054814c0a0 con 0x7f05480bc9b0 2023-12-16T21:43:17.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.363+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0548067e60 con 0x7f05480bc9b0 2023-12-16T21:43:17.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.367+0000 7f05457fa640 1 -- 172.21.15.46:0/1199897750 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f0530012990 con 0x7f05480bc9b0 2023-12-16T21:43:17.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.367+0000 7f05457fa640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0528066470 0x7f0528068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.374 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.367+0000 7f05457fa640 1 -- 172.21.15.46:0/1199897750 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0530083770 con 0x7f05480bc9b0 2023-12-16T21:43:17.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.367+0000 7f0547fff640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0528066470 0x7f0528068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.375 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.367+0000 7f0547fff640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0528066470 0x7f0528068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0538003fd0 tx=0x7f05380039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.378 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.371+0000 7f05457fa640 1 -- 172.21.15.46:0/1199897750 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f0530050f30 con 0x7f05480bc9b0 2023-12-16T21:43:17.535 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.527+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f054814c5e0 con 0x7f05480bc9b0 2023-12-16T21:43:17.536 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.527+0000 7f05457fa640 1 -- 172.21.15.46:0/1199897750 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v26) v1 ==== 84+0+65 (secure 0 0 0) 0x7f054814c5e0 con 0x7f05480bc9b0 2023-12-16T21:43:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:17.537 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2023-12-16T21:43:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: { 2023-12-16T21:43:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2023-12-16T21:43:17.537 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2023-12-16T21:43:17.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:17.538 INFO:teuthology.orchestra.run.smithi046.stdout:] 2023-12-16T21:43:17.539 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0528066470 msgr2=0x7f0528068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0528066470 0x7f0528068930 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0538003fd0 tx=0x7f05380039a0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05480bc9b0 msgr2=0x7f054814d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05480bc9b0 0x7f054814d710 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f053000c4d0 tx=0x7f053000c990 comp rx=0 tx=0).stop 2023-12-16T21:43:17.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 shutdown_connections 2023-12-16T21:43:17.540 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0528066470 0x7f0528068930 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f054814e0b0 0x7f054814c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.541 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f05480bc9b0 0x7f054814d710 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 --2- 172.21.15.46:0/1199897750 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f05480bc5d0 0x7f054813e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 >> 172.21.15.46:0/1199897750 conn(0x7f0548072340 msgr2=0x7f05481490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 shutdown_connections 2023-12-16T21:43:17.542 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.531+0000 7f054e413640 1 -- 172.21.15.46:0/1199897750 wait complete. 2023-12-16T21:43:17.555 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-16T21:43:17.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-16T21:43:17.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf758a640 1 Processor -- start 2023-12-16T21:43:17.682 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf758a640 1 -- start start 2023-12-16T21:43:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf758a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cf006c920 0x7f0cf00aff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf758a640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf00b3cd0 con 0x7f0cf0145170 2023-12-16T21:43:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf758a640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf00b1ea0 con 0x7f0cf00afa00 2023-12-16T21:43:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf758a640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf00b1b70 con 0x7f0cf006c920 2023-12-16T21:43:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf5326640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cf006c920 0x7f0cf00aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf5326640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0cf006c920 0x7f0cf00aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52152/0 (socket says 172.21.15.46:52152) 2023-12-16T21:43:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf5326640 1 -- 172.21.15.46:0/2706839560 learned_addr learned my addr 172.21.15.46:0/2706839560 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf5b27640 1 --1- 172.21.15.46:0/2706839560 >> v1:172.21.15.46:6789/0 conn(0x7f0cf0145170 0x7f0cf014ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46638/0 (socket says 172.21.15.46:46638) 2023-12-16T21:43:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf5326640 1 -- 172.21.15.46:0/2706839560 >> v1:172.21.15.161:6789/0 conn(0x7f0cf00afa00 legacy=0x7f0cf0144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf5326640 1 -- 172.21.15.46:0/2706839560 >> v1:172.21.15.46:6789/0 conn(0x7f0cf0145170 legacy=0x7f0cf014ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0ce7fff640 1 -- 172.21.15.46:0/2706839560 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30694069 0 0) 0x7f0cf00b3cd0 con 0x7f0cf0145170 2023-12-16T21:43:17.684 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.675+0000 7f0cf5326640 1 -- 172.21.15.46:0/2706839560 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf006cd00 con 0x7f0cf006c920 2023-12-16T21:43:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5326640 1 --2- 172.21.15.46:0/2706839560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf006c920 0x7f0cf00aff90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009870 tx=0x7f0ce001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7d340963d54877 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0ce7fff640 1 -- 172.21.15.46:0/2706839560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00ab620 con 0x7f0cf006c920 2023-12-16T21:43:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0ce7fff640 1 -- 172.21.15.46:0/2706839560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce00ab780 con 0x7f0cf006c920 2023-12-16T21:43:17.685 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0ce7fff640 1 -- 172.21.15.46:0/2706839560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce00abb10 con 0x7f0cf006c920 2023-12-16T21:43:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- 172.21.15.46:0/2706839560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf006c920 msgr2=0x7f0cf00aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 --2- 172.21.15.46:0/2706839560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf006c920 0x7f0cf00aff90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009870 tx=0x7f0ce001c510 comp rx=0 tx=0).stop 2023-12-16T21:43:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- 172.21.15.46:0/2706839560 shutdown_connections 2023-12-16T21:43:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 --2- 172.21.15.46:0/2706839560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf006c920 0x7f0cf00aff90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.686 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- 172.21.15.46:0/2706839560 >> 172.21.15.46:0/2706839560 conn(0x7f0cf00b4890 msgr2=0x7f0cf00b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- 172.21.15.46:0/2706839560 shutdown_connections 2023-12-16T21:43:17.687 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- 172.21.15.46:0/2706839560 wait complete. 2023-12-16T21:43:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 Processor -- start 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- start start 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf006c920 0x7f0cf0135e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cf00afa00 0x7f0cf006e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf013f3e0 0x7f0cf006eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf013f860 con 0x7f0cf013f3e0 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf013fb40 con 0x7f0cf006c920 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0141620 con 0x7f0cf00afa00 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf013f3e0 0x7f0cf006eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf013f3e0 0x7f0cf006eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52164/0 (socket says 172.21.15.46:52164) 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 -- 172.21.15.46:0/3381910339 learned_addr learned my addr 172.21.15.46:0/3381910339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf4b25640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cf00afa00 0x7f0cf006e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5326640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf006c920 0x7f0cf0135e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 -- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cf00afa00 msgr2=0x7f0cf006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cf00afa00 0x7f0cf006e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 -- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf006c920 msgr2=0x7f0cf0135e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf006c920 0x7f0cf0135e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 -- 172.21.15.46:0/3381910339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00094d0 con 0x7f0cf013f3e0 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5326640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf006c920 0x7f0cf0135e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf5b27640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf013f3e0 0x7f0cf006eee0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8001570 tx=0x7f0ce800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0ce5ffb640 1 -- 172.21.15.46:0/3381910339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800a640 con 0x7f0cf013f3e0 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0ce5ffb640 1 -- 172.21.15.46:0/3381910339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce800a7c0 con 0x7f0cf013f3e0 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0ce5ffb640 1 -- 172.21.15.46:0/3381910339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800ab70 con 0x7f0cf013f3e0 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.679+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf01379b0 con 0x7f0cf013f3e0 2023-12-16T21:43:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.683+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf0141c80 con 0x7f0cf013f3e0 2023-12-16T21:43:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.683+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf0141840 con 0x7f0cf013f3e0 2023-12-16T21:43:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.683+0000 7f0ce5ffb640 1 -- 172.21.15.46:0/3381910339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f0ce800acd0 con 0x7f0cf013f3e0 2023-12-16T21:43:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.683+0000 7f0ce5ffb640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd40664a0 0x7f0cd4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.683+0000 7f0ce5ffb640 1 -- 172.21.15.46:0/3381910339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0ce8083140 con 0x7f0cf013f3e0 2023-12-16T21:43:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.683+0000 7f0cf5326640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd40664a0 0x7f0cd4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:17.692 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.683+0000 7f0cf5326640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd40664a0 0x7f0cd4068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00af000 tx=0x7f0ce0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.687+0000 7f0ce5ffb640 1 -- 172.21.15.46:0/3381910339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f0ce80507f0 con 0x7f0cf013f3e0 2023-12-16T21:43:17.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:17.843+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0cf00689d0 con 0x7f0cf013f3e0 2023-12-16T21:43:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.235+0000 7f0ce5ffb640 1 -- 172.21.15.46:0/3381910339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0ce8053e10 con 0x7f0cf013f3e0 2023-12-16T21:43:18.242 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2023-12-16T21:43:18.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd40664a0 msgr2=0x7f0cd4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:18.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd40664a0 0x7f0cd4068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00af000 tx=0x7f0ce0008040 comp rx=0 tx=0).stop 2023-12-16T21:43:18.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf013f3e0 msgr2=0x7f0cf006eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf013f3e0 0x7f0cf006eee0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8001570 tx=0x7f0ce800cca0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 shutdown_connections 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0cd40664a0 0x7f0cd4068960 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cf013f3e0 0x7f0cf006eee0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0cf00afa00 0x7f0cf006e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 --2- 172.21.15.46:0/3381910339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0cf006c920 0x7f0cf0135e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 >> 172.21.15.46:0/3381910339 conn(0x7f0cf00b4890 msgr2=0x7f0cf00b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 shutdown_connections 2023-12-16T21:43:18.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.239+0000 7f0cf758a640 1 -- 172.21.15.46:0/3381910339 wait complete. 2023-12-16T21:43:18.270 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-16T21:43:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.395+0000 7ff644b4d640 1 Processor -- start 2023-12-16T21:43:18.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.395+0000 7ff644b4d640 1 -- start start 2023-12-16T21:43:18.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff644b4d640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff6400b5f80 0x7ff6400b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:18.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff644b4d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6400b3e20 0x7ff640147c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff644b4d640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6400b4280 con 0x7ff6401481a0 2023-12-16T21:43:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff644b4d640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6400b1f80 con 0x7ff6400b3e20 2023-12-16T21:43:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff644b4d640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff640148610 con 0x7ff6400b5f80 2023-12-16T21:43:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6400b3e20 0x7ff640147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff6400b3e20 0x7ff640147c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52174/0 (socket says 172.21.15.46:52174) 2023-12-16T21:43:18.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 -- 172.21.15.46:0/789113321 learned_addr learned my addr 172.21.15.46:0/789113321 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff63e59c640 1 --2- 172.21.15.46:0/789113321 >> v2:172.21.15.161:3300/0 conn(0x7ff6400b5f80 0x7ff6400b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 -- 172.21.15.46:0/789113321 >> v2:172.21.15.161:3300/0 conn(0x7ff6400b5f80 msgr2=0x7ff6400b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 --2- 172.21.15.46:0/789113321 >> v2:172.21.15.161:3300/0 conn(0x7ff6400b5f80 0x7ff6400b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 -- 172.21.15.46:0/789113321 >> v1:172.21.15.161:6789/0 conn(0x7ff6401481a0 legacy=0x7ff6400708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:18.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 -- 172.21.15.46:0/789113321 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6400b6360 con 0x7ff6400b3e20 2023-12-16T21:43:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff637fff640 1 --2- 172.21.15.46:0/789113321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b3e20 0x7ff640147c60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff640071f50 tx=0x7ff62801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102afdc52c497c2e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff63dd9b640 1 -- 172.21.15.46:0/789113321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280ab600 con 0x7ff6400b3e20 2023-12-16T21:43:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff63dd9b640 1 -- 172.21.15.46:0/789113321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280a7050 con 0x7ff6400b3e20 2023-12-16T21:43:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff63dd9b640 1 -- 172.21.15.46:0/789113321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280ab980 con 0x7ff6400b3e20 2023-12-16T21:43:18.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff644b4d640 1 -- 172.21.15.46:0/789113321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b3e20 msgr2=0x7ff640147c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.399+0000 7ff644b4d640 1 --2- 172.21.15.46:0/789113321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b3e20 0x7ff640147c60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff640071f50 tx=0x7ff62801c4c0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- 172.21.15.46:0/789113321 shutdown_connections 2023-12-16T21:43:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 --2- 172.21.15.46:0/789113321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b3e20 0x7ff640147c60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 --2- 172.21.15.46:0/789113321 >> v2:172.21.15.161:3300/0 conn(0x7ff6400b5f80 0x7ff6400b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- 172.21.15.46:0/789113321 >> 172.21.15.46:0/789113321 conn(0x7ff64006a980 msgr2=0x7ff64006ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- 172.21.15.46:0/789113321 shutdown_connections 2023-12-16T21:43:18.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- 172.21.15.46:0/789113321 wait complete. 2023-12-16T21:43:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 Processor -- start 2023-12-16T21:43:18.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- start start 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6400b3e20 0x7ff64013ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b5f80 0x7ff640134af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6401481a0 0x7ff6401448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401404a0 con 0x7ff6400b5f80 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6401e0ec0 con 0x7ff6400b3e20 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64013c9e0 con 0x7ff6401481a0 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b5f80 0x7ff640134af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b5f80 0x7ff640134af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52182/0 (socket says 172.21.15.46:52182) 2023-12-16T21:43:18.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 -- 172.21.15.46:0/777710743 learned_addr learned my addr 172.21.15.46:0/777710743 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff63e59c640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6400b3e20 0x7ff64013ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff63ed9d640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6401481a0 0x7ff6401448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 -- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6401481a0 msgr2=0x7ff6401448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6401481a0 0x7ff6401448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 -- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6400b3e20 msgr2=0x7ff64013ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6400b3e20 0x7ff64013ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 -- 172.21.15.46:0/777710743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6280094d0 con 0x7ff6400b5f80 2023-12-16T21:43:18.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff637fff640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b5f80 0x7ff640134af0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff62801a0f0 tx=0x7ff628019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff636ffd640 1 -- 172.21.15.46:0/777710743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b6650 con 0x7ff6400b5f80 2023-12-16T21:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff636ffd640 1 -- 172.21.15.46:0/777710743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280b67d0 con 0x7ff6400b5f80 2023-12-16T21:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff636ffd640 1 -- 172.21.15.46:0/777710743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280b6b80 con 0x7ff6400b5f80 2023-12-16T21:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6401e0870 con 0x7ff6400b5f80 2023-12-16T21:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.403+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64013f4a0 con 0x7ff6400b5f80 2023-12-16T21:43:18.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.407+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6400b1790 con 0x7ff6400b5f80 2023-12-16T21:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.407+0000 7ff636ffd640 1 -- 172.21.15.46:0/777710743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7ff6280b2040 con 0x7ff6400b5f80 2023-12-16T21:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.407+0000 7ff636ffd640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff6200664a0 0x7ff620068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.407+0000 7ff636ffd640 1 -- 172.21.15.46:0/777710743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff62811e4d0 con 0x7ff6400b5f80 2023-12-16T21:43:18.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.407+0000 7ff63e59c640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff6200664a0 0x7ff620068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:18.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.407+0000 7ff63e59c640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff6200664a0 0x7ff620068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff630007a60 tx=0x7ff630008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:18.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.411+0000 7ff636ffd640 1 -- 172.21.15.46:0/777710743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7ff6280ebc50 con 0x7ff6400b5f80 2023-12-16T21:43:18.576 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:18.567+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff6401e06d0 con 0x7ff6400b5f80 2023-12-16T21:43:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.239+0000 7ff636ffd640 1 -- 172.21.15.46:0/777710743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v28) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff6401e06d0 con 0x7ff6400b5f80 2023-12-16T21:43:19.247 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2023-12-16T21:43:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff6200664a0 msgr2=0x7ff620068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff6200664a0 0x7ff620068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff630007a60 tx=0x7ff630008040 comp rx=0 tx=0).stop 2023-12-16T21:43:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b5f80 msgr2=0x7ff640134af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:19.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b5f80 0x7ff640134af0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff62801a0f0 tx=0x7ff628019260 comp rx=0 tx=0).stop 2023-12-16T21:43:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 shutdown_connections 2023-12-16T21:43:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff6200664a0 0x7ff620068960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff6401481a0 0x7ff6401448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff6400b5f80 0x7ff640134af0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 --2- 172.21.15.46:0/777710743 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff6400b3e20 0x7ff64013ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 >> 172.21.15.46:0/777710743 conn(0x7ff64006a980 msgr2=0x7ff6400b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 shutdown_connections 2023-12-16T21:43:19.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.243+0000 7ff644b4d640 1 -- 172.21.15.46:0/777710743 wait complete. 2023-12-16T21:43:19.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-16T21:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd72b55d640 1 Processor -- start 2023-12-16T21:43:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd72b55d640 1 -- start start 2023-12-16T21:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd72b55d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 0x7fd7240aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd72b55d640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240b1730 con 0x7fd7241451c0 2023-12-16T21:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd72b55d640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72406c7f0 con 0x7fd7240af300 2023-12-16T21:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd72b55d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72406cd70 con 0x7fd7240b1eb0 2023-12-16T21:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd729afa640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fd7241451c0 0x7fd72414ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42444/0 (socket says 172.21.15.46:42444) 2023-12-16T21:43:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd729afa640 1 -- 172.21.15.46:0/3578150348 learned_addr learned my addr 172.21.15.46:0/3578150348 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd7292f9640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 0x7fd7240aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626601913 0 0) 0x7fd7240b1730 con 0x7fd7241451c0 2023-12-16T21:43:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd710003530 con 0x7fd7241451c0 2023-12-16T21:43:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1249885360 0 0) 0x7fd710003530 con 0x7fd7241451c0 2023-12-16T21:43:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 msgr2=0x7fd7240aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 0x7fd7240aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 >> v1:172.21.15.161:6789/0 conn(0x7fd7240af300 legacy=0x7fd724144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240b2290 con 0x7fd7241451c0 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fd720001680 con 0x7fd7241451c0 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 >> v1:172.21.15.46:6789/0 conn(0x7fd7241451c0 legacy=0x7fd72414ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7100039b0 0x7fd710003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7100047d0 0x7fd710004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 0x7fd710005260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72406cd70 con 0x7fd7100039b0 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000bb50 con 0x7fd7100047d0 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71000bd10 con 0x7fd7240b1eb0 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd7292f9640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7100039b0 0x7fd710003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd728af8640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7100047d0 0x7fd710004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd729afa640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 0x7fd710005260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd7292f9640 1 -- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 msgr2=0x7fd710005260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd7292f9640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7240b1eb0 0x7fd710005260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd7292f9640 1 -- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7100047d0 msgr2=0x7fd710004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd7292f9640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7100047d0 0x7fd710004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd7292f9640 1 -- 172.21.15.46:0/3578150348 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd710003530 con 0x7fd7100039b0 2023-12-16T21:43:19.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.407+0000 7fd728af8640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd7100047d0 0x7fd710004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd7292f9640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7100039b0 0x7fd710003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009fc0 tx=0x7fd71c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed767d684692591 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0a5a60 con 0x7fd7100039b0 2023-12-16T21:43:19.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c00f040 con 0x7fd7100039b0 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd71bfff640 1 -- 172.21.15.46:0/3578150348 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c0a5e00 con 0x7fd7100039b0 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 -- 172.21.15.46:0/3578150348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7100039b0 msgr2=0x7fd710003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 --2- 172.21.15.46:0/3578150348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd7100039b0 0x7fd710003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009fc0 tx=0x7fd71c01cc60 comp rx=0 tx=0).stop 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd729afa640 1 -- 172.21.15.46:0/3578150348 reap_dead start 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 -- 172.21.15.46:0/3578150348 shutdown_connections 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 -- 172.21.15.46:0/3578150348 >> 172.21.15.46:0/3578150348 conn(0x7fd7240b4900 msgr2=0x7fd7240b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 -- 172.21.15.46:0/3578150348 shutdown_connections 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 -- 172.21.15.46:0/3578150348 wait complete. 2023-12-16T21:43:19.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 Processor -- start 2023-12-16T21:43:19.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.411+0000 7fd72b55d640 1 -- start start 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd724141210 0x7fd724099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7241dcc50 0x7fd7241dd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd72409cb20 0x7fd72409cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7241415f0 con 0x7fd72409cb20 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72414ff00 con 0x7fd724141210 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7241481b0 con 0x7fd7241dcc50 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd72409cb20 0x7fd72409cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd72409cb20 0x7fd72409cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53656/0 (socket says 172.21.15.46:53656) 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 -- 172.21.15.46:0/190937730 learned_addr learned my addr 172.21.15.46:0/190937730 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd728af8640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7241dcc50 0x7fd7241dd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd7292f9640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd724141210 0x7fd724099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 -- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7241dcc50 msgr2=0x7fd7241dd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7241dcc50 0x7fd7241dd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 -- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd724141210 msgr2=0x7fd724099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd724141210 0x7fd724099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 -- 172.21.15.46:0/190937730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72409a0d0 con 0x7fd72409cb20 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd7292f9640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd724141210 0x7fd724099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd729afa640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd72409cb20 0x7fd72409cf80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7240b5920 tx=0x7fd72000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd719ffb640 1 -- 172.21.15.46:0/190937730 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72000eed0 con 0x7fd72409cb20 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd719ffb640 1 -- 172.21.15.46:0/190937730 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd72000c590 con 0x7fd72409cb20 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7241dd610 con 0x7fd72409cb20 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd719ffb640 1 -- 172.21.15.46:0/190937730 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72000c960 con 0x7fd72409cb20 2023-12-16T21:43:19.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd724147530 con 0x7fd72409cb20 2023-12-16T21:43:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd724068960 con 0x7fd72409cb20 2023-12-16T21:43:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd719ffb640 1 -- 172.21.15.46:0/190937730 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fd72000cac0 con 0x7fd72409cb20 2023-12-16T21:43:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd719ffb640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd708066470 0x7fd708068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:19.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd719ffb640 1 -- 172.21.15.46:0/190937730 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd7200838f0 con 0x7fd72409cb20 2023-12-16T21:43:19.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.415+0000 7fd7292f9640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd708066470 0x7fd708068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:19.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.419+0000 7fd7292f9640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd708066470 0x7fd708068930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0af000 tx=0x7fd71c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:19.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.423+0000 7fd719ffb640 1 -- 172.21.15.46:0/190937730 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fd720050f20 con 0x7fd72409cb20 2023-12-16T21:43:19.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:19.611+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd7240b69b0 con 0x7fd72409cb20 2023-12-16T21:43:20.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.311+0000 7fd719ffb640 1 -- 172.21.15.46:0/190937730 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd720054540 con 0x7fd72409cb20 2023-12-16T21:43:20.321 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 4 and data pool 5 2023-12-16T21:43:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd708066470 msgr2=0x7fd708068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd708066470 0x7fd708068930 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd71c0af000 tx=0x7fd71c0a7040 comp rx=0 tx=0).stop 2023-12-16T21:43:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd72409cb20 msgr2=0x7fd72409cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd72409cb20 0x7fd72409cf80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd7240b5920 tx=0x7fd72000eb90 comp rx=0 tx=0).stop 2023-12-16T21:43:20.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 shutdown_connections 2023-12-16T21:43:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd708066470 0x7fd708068930 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd72409cb20 0x7fd72409cf80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd7241dcc50 0x7fd7241dd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 --2- 172.21.15.46:0/190937730 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd724141210 0x7fd724099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 >> 172.21.15.46:0/190937730 conn(0x7fd7240b4900 msgr2=0x7fd7240b6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 shutdown_connections 2023-12-16T21:43:20.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.319+0000 7fd72b55d640 1 -- 172.21.15.46:0/190937730 wait complete. 2023-12-16T21:43:20.361 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 Processor -- start 2023-12-16T21:43:20.477 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- start start 2023-12-16T21:43:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f068006c920 0x7f06800afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06800af2e0 con 0x7f068009dd50 2023-12-16T21:43:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06800b1f50 con 0x7f06800b3fd0 2023-12-16T21:43:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068006cd00 con 0x7f068006c920 2023-12-16T21:43:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06851f5640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f068009dd50 0x7f068014ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42454/0 (socket says 172.21.15.46:42454) 2023-12-16T21:43:20.478 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06851f5640 1 -- 172.21.15.46:0/1864248372 learned_addr learned my addr 172.21.15.46:0/1864248372 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 --2- 172.21.15.46:0/1864248372 >> v2:172.21.15.161:3300/0 conn(0x7f068006c920 0x7f06800afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141925099 0 0) 0x7f06800af2e0 con 0x7f068009dd50 2023-12-16T21:43:20.479 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0670003530 con 0x7f068009dd50 2023-12-16T21:43:20.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3308615326 0 0) 0x7f0670003530 con 0x7f068009dd50 2023-12-16T21:43:20.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 >> v2:172.21.15.161:3300/0 conn(0x7f068006c920 msgr2=0x7f06800afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:20.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 --2- 172.21.15.46:0/1864248372 >> v2:172.21.15.161:3300/0 conn(0x7f068006c920 0x7f06800afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 >> v1:172.21.15.161:6789/0 conn(0x7f06800b3fd0 legacy=0x7f068013e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06800af530 con 0x7f068009dd50 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f0678001a00 con 0x7f068009dd50 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 >> v1:172.21.15.46:6789/0 conn(0x7f068009dd50 legacy=0x7f068014ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0670003980 0x7f0670003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06700047d0 0x7f0670004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0670005190 0x7f067000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068006cd00 con 0x7f0670003980 2023-12-16T21:43:20.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f067000c130 con 0x7f06700047d0 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f067000c2f0 con 0x7f0670005190 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0670003980 0x7f0670003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067ffff640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06700047d0 0x7f0670004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 -- 172.21.15.46:0/1864248372 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0670005190 msgr2=0x7f067000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0670005190 0x7f067000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 -- 172.21.15.46:0/1864248372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06700047d0 msgr2=0x7f0670004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f06700047d0 0x7f0670004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 -- 172.21.15.46:0/1864248372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0670003530 con 0x7f0670003980 2023-12-16T21:43:20.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06851f5640 1 -- 172.21.15.46:0/1864248372 reap_dead start 2023-12-16T21:43:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f06849f4640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0670003980 0x7f0670003da0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0674009ae0 tx=0x7f067401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdfa12e68211add server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740a6430 con 0x7f0670003980 2023-12-16T21:43:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06740a6590 con 0x7f0670003980 2023-12-16T21:43:20.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f067f7fe640 1 -- 172.21.15.46:0/1864248372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06740a6940 con 0x7f0670003980 2023-12-16T21:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- 172.21.15.46:0/1864248372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0670003980 msgr2=0x7f0670003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0670003980 0x7f0670003da0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0674009ae0 tx=0x7f067401ca50 comp rx=0 tx=0).stop 2023-12-16T21:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- 172.21.15.46:0/1864248372 shutdown_connections 2023-12-16T21:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 --2- 172.21.15.46:0/1864248372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0670003980 0x7f0670003da0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- 172.21.15.46:0/1864248372 >> 172.21.15.46:0/1864248372 conn(0x7f06800b48e0 msgr2=0x7f06800b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- 172.21.15.46:0/1864248372 shutdown_connections 2023-12-16T21:43:20.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- 172.21.15.46:0/1864248372 wait complete. 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 Processor -- start 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.475+0000 7f0686c58640 1 -- start start 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0670003980 0x7f06801413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680147e50 0x7f0680140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06801482d0 0x7f0680140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0680140a90 con 0x7f0680147e50 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068006f030 con 0x7f0670003980 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06801dfd30 con 0x7f06801482d0 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680147e50 0x7f0680140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680147e50 0x7f0680140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53684/0 (socket says 172.21.15.46:53684) 2023-12-16T21:43:20.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 -- 172.21.15.46:0/569975494 learned_addr learned my addr 172.21.15.46:0/569975494 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f06851f5640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06801482d0 0x7f0680140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f06849f4640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0670003980 0x7f06801413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 -- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06801482d0 msgr2=0x7f0680140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06801482d0 0x7f0680140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 -- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0670003980 msgr2=0x7f06801413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0670003980 0x7f06801413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 -- 172.21.15.46:0/569975494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06801e0010 con 0x7f0680147e50 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067ffff640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680147e50 0x7f0680140010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f066c00e4d0 tx=0x7f066c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067d7fa640 1 -- 172.21.15.46:0/569975494 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0053a0 con 0x7f0680147e50 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067d7fa640 1 -- 172.21.15.46:0/569975494 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c003600 con 0x7f0680147e50 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068006f310 con 0x7f0680147e50 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067d7fa640 1 -- 172.21.15.46:0/569975494 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c003990 con 0x7f0680147e50 2023-12-16T21:43:20.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06801e02f0 con 0x7f0680147e50 2023-12-16T21:43:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06800679e0 con 0x7f0680147e50 2023-12-16T21:43:20.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067d7fa640 1 -- 172.21.15.46:0/569975494 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f066c003af0 con 0x7f0680147e50 2023-12-16T21:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067d7fa640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f06640664a0 0x7f0664068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f067d7fa640 1 -- 172.21.15.46:0/569975494 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f066c0838d0 con 0x7f0680147e50 2023-12-16T21:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.479+0000 7f06849f4640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f06640664a0 0x7f0664068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.489 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.483+0000 7f06849f4640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f06640664a0 0x7f0664068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06740ad000 tx=0x7f06740046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:20.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.483+0000 7f067d7fa640 1 -- 172.21.15.46:0/569975494 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f066c050d50 con 0x7f0680147e50 2023-12-16T21:43:20.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.643+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06800688f0 con 0x7f0680147e50 2023-12-16T21:43:20.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.647+0000 7f067d7fa640 1 -- 172.21.15.46:0/569975494 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f066c054370 con 0x7f0680147e50 2023-12-16T21:43:20.651 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:19.617456+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:43:17.887506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-16T21:43:18.612695+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f06640664a0 msgr2=0x7f0664068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f06640664a0 0x7f0664068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f06740ad000 tx=0x7f06740046d0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680147e50 msgr2=0x7f0680140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680147e50 0x7f0680140010 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f066c00e4d0 tx=0x7f066c00e990 comp rx=0 tx=0).stop 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 shutdown_connections 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f06640664a0 0x7f0664068960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f06801482d0 0x7f0680140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0680147e50 0x7f0680140010 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 --2- 172.21.15.46:0/569975494 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0670003980 0x7f06801413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 >> 172.21.15.46:0/569975494 conn(0x7f06800b48e0 msgr2=0x7f06800b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 shutdown_connections 2023-12-16T21:43:20.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.651+0000 7f0686c58640 1 -- 172.21.15.46:0/569975494 wait complete. 2023-12-16T21:43:20.672 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.795+0000 7fc00903b640 1 Processor -- start 2023-12-16T21:43:20.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.795+0000 7fc00903b640 1 -- start start 2023-12-16T21:43:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.795+0000 7fc00903b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0040af2e0 0x7fc004147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.795+0000 7fc00903b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0040ad7c0 0x7fc0041475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.795+0000 7fc00903b640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040af770 con 0x7fc00406c920 2023-12-16T21:43:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.795+0000 7fc00903b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0040afc20 con 0x7fc0040af2e0 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.795+0000 7fc00903b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040aeea0 con 0x7fc0040ad7c0 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0040af2e0 0x7fc004147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc0040af2e0 0x7fc004147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53696/0 (socket says 172.21.15.46:53696) 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 -- 172.21.15.46:0/737297457 learned_addr learned my addr 172.21.15.46:0/737297457 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00359e640 1 --2- 172.21.15.46:0/737297457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0040ad7c0 0x7fc0041475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 -- 172.21.15.46:0/737297457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0040ad7c0 msgr2=0x7fc0041475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 --2- 172.21.15.46:0/737297457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0040ad7c0 0x7fc0041475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 -- 172.21.15.46:0/737297457 >> v1:172.21.15.161:6789/0 conn(0x7fc00406c920 legacy=0x7fc0040ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 -- 172.21.15.46:0/737297457 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004147b00 con 0x7fc0040af2e0 2023-12-16T21:43:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fbffa59c640 1 --2- 172.21.15.46:0/737297457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040af2e0 0x7fc004147080 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009870 tx=0x7fbfec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56af2edcb3a085c server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00259c640 1 -- 172.21.15.46:0/737297457 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0ab620 con 0x7fc0040af2e0 2023-12-16T21:43:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00259c640 1 -- 172.21.15.46:0/737297457 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0ab780 con 0x7fc0040af2e0 2023-12-16T21:43:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00259c640 1 -- 172.21.15.46:0/737297457 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0abb10 con 0x7fc0040af2e0 2023-12-16T21:43:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 -- 172.21.15.46:0/737297457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040af2e0 msgr2=0x7fc004147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 --2- 172.21.15.46:0/737297457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040af2e0 0x7fc004147080 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009870 tx=0x7fbfec01c420 comp rx=0 tx=0).stop 2023-12-16T21:43:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 -- 172.21.15.46:0/737297457 shutdown_connections 2023-12-16T21:43:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 --2- 172.21.15.46:0/737297457 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc0040ad7c0 0x7fc0041475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 --2- 172.21.15.46:0/737297457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040af2e0 0x7fc004147080 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 -- 172.21.15.46:0/737297457 >> 172.21.15.46:0/737297457 conn(0x7fc0040b48e0 msgr2=0x7fc0040b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 -- 172.21.15.46:0/737297457 shutdown_connections 2023-12-16T21:43:20.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.799+0000 7fc00903b640 1 -- 172.21.15.46:0/737297457 wait complete. 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 Processor -- start 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 -- start start 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00406c920 0x7fc004145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ad7c0 0x7fc0041dd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc004141b40 0x7fc004141fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00409a7e0 con 0x7fc0040ad7c0 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc004143c80 con 0x7fc004141b40 2023-12-16T21:43:20.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc004143e60 con 0x7fc00406c920 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ad7c0 0x7fc0041dd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ad7c0 0x7fc0041dd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53702/0 (socket says 172.21.15.46:53702) 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 -- 172.21.15.46:0/1918299286 learned_addr learned my addr 172.21.15.46:0/1918299286 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc002d9d640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00406c920 0x7fc004145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00359e640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc004141b40 0x7fc004141fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 -- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00406c920 msgr2=0x7fc004145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00406c920 0x7fc004145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 -- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc004141b40 msgr2=0x7fc004141fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc004141b40 0x7fc004141fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 -- 172.21.15.46:0/1918299286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfec0094d0 con 0x7fc0040ad7c0 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffa59c640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ad7c0 0x7fc0041dd420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbfec01aad0 tx=0x7fbfec01ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffbfff640 1 -- 172.21.15.46:0/1918299286 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0be5f0 con 0x7fc0040ad7c0 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffbfff640 1 -- 172.21.15.46:0/1918299286 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfec0be770 con 0x7fc0040ad7c0 2023-12-16T21:43:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00409aca0 con 0x7fc0040ad7c0 2023-12-16T21:43:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fbffbfff640 1 -- 172.21.15.46:0/1918299286 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfec0beb20 con 0x7fc0040ad7c0 2023-12-16T21:43:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0041dcf00 con 0x7fc0040ad7c0 2023-12-16T21:43:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.803+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0040679e0 con 0x7fc0040ad7c0 2023-12-16T21:43:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.807+0000 7fbffbfff640 1 -- 172.21.15.46:0/1918299286 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fbfec0b43f0 con 0x7fc0040ad7c0 2023-12-16T21:43:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.807+0000 7fbffbfff640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbfe4066470 0x7fbfe4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.807+0000 7fbffbfff640 1 -- 172.21.15.46:0/1918299286 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfec11ed00 con 0x7fc0040ad7c0 2023-12-16T21:43:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.807+0000 7fc002d9d640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbfe4066470 0x7fbfe4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.807+0000 7fc002d9d640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbfe4066470 0x7fbfe4068930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc00413c6e0 tx=0x7fbff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.811+0000 7fbffbfff640 1 -- 172.21.15.46:0/1918299286 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fbfec0ec170 con 0x7fc0040ad7c0 2023-12-16T21:43:20.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.967+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0041dca80 con 0x7fc0040ad7c0 2023-12-16T21:43:20.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fbffbfff640 1 -- 172.21.15.46:0/1918299286 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+16582 (secure 0 0 0) 0x7fc0041dca80 con 0x7fc0040ad7c0 2023-12-16T21:43:20.973 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:20.973 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":29,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:19.617456+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:43:17.887506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-16T21:43:18.612695+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbfe4066470 msgr2=0x7fbfe4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbfe4066470 0x7fbfe4068930 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc00413c6e0 tx=0x7fbff4009040 comp rx=0 tx=0).stop 2023-12-16T21:43:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ad7c0 msgr2=0x7fc0041dd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:20.975 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ad7c0 0x7fc0041dd420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbfec01aad0 tx=0x7fbfec01ac60 comp rx=0 tx=0).stop 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 shutdown_connections 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbfe4066470 0x7fbfe4068930 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc004141b40 0x7fc004141fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc0040ad7c0 0x7fc0041dd420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 --2- 172.21.15.46:0/1918299286 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc00406c920 0x7fc004145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 >> 172.21.15.46:0/1918299286 conn(0x7fc0040b48e0 msgr2=0x7fc0040b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 shutdown_connections 2023-12-16T21:43:20.976 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:20.971+0000 7fc00903b640 1 -- 172.21.15.46:0/1918299286 wait complete. 2023-12-16T21:43:20.993 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-16T21:43:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820fdcb640 1 Processor -- start 2023-12-16T21:43:21.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820fdcb640 1 -- start start 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820fdcb640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f82080b1eb0 0x7f82080aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820fdcb640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82081451c0 0x7f820814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820fdcb640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82080b1730 con 0x7f82080af300 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820fdcb640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820806cd70 con 0x7f82080b1eb0 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820fdcb640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82080b0ad0 con 0x7f82081451c0 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820e368640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82081451c0 0x7f820814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820db67640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f82080b1eb0 0x7f82080aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820e368640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82081451c0 0x7f820814ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:48108/0 (socket says 172.21.15.46:48108) 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820db67640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f82080b1eb0 0x7f82080aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:59856/0 (socket says 172.21.15.46:59856) 2023-12-16T21:43:21.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820e368640 1 -- 172.21.15.46:0/771702072 learned_addr learned my addr 172.21.15.46:0/771702072 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820db67640 1 -- 172.21.15.46:0/771702072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82081451c0 msgr2=0x7f820814ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820db67640 1 --2- 172.21.15.46:0/771702072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82081451c0 0x7f820814ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820db67640 1 -- 172.21.15.46:0/771702072 >> v1:172.21.15.161:6789/0 conn(0x7f82080af300 legacy=0x7f8208144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.119+0000 7f820db67640 1 -- 172.21.15.46:0/771702072 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820806c7f0 con 0x7f82080b1eb0 2023-12-16T21:43:21.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820db67640 1 --2- 172.21.15.46:0/771702072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f82080aff30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82040097c0 tx=0x7f820401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c004042f0f509809 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:21.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820cb65640 1 -- 172.21.15.46:0/771702072 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040ad660 con 0x7f82080b1eb0 2023-12-16T21:43:21.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820cb65640 1 -- 172.21.15.46:0/771702072 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82040ad7c0 con 0x7f82080b1eb0 2023-12-16T21:43:21.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820cb65640 1 -- 172.21.15.46:0/771702072 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82040adb50 con 0x7f82080b1eb0 2023-12-16T21:43:21.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 -- 172.21.15.46:0/771702072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 msgr2=0x7f82080aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 --2- 172.21.15.46:0/771702072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f82080aff30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f82040097c0 tx=0x7f820401c360 comp rx=0 tx=0).stop 2023-12-16T21:43:21.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 -- 172.21.15.46:0/771702072 shutdown_connections 2023-12-16T21:43:21.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 --2- 172.21.15.46:0/771702072 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82081451c0 0x7f820814ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.127 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 --2- 172.21.15.46:0/771702072 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f82080aff30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 -- 172.21.15.46:0/771702072 >> 172.21.15.46:0/771702072 conn(0x7f82080b4900 msgr2=0x7f82080b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:21.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 -- 172.21.15.46:0/771702072 shutdown_connections 2023-12-16T21:43:21.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 -- 172.21.15.46:0/771702072 wait complete. 2023-12-16T21:43:21.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 Processor -- start 2023-12-16T21:43:21.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.123+0000 7f820fdcb640 1 -- start start 2023-12-16T21:43:21.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82080af300 0x7f820813ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f8208070940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82081451c0 0x7f82081366e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8208136250 con 0x7f82081451c0 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82081ddba0 con 0x7f82080b1eb0 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82081dde80 con 0x7f82080af300 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82081451c0 0x7f82081366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82081451c0 0x7f82081366e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53718/0 (socket says 172.21.15.46:53718) 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 -- 172.21.15.46:0/372932493 learned_addr learned my addr 172.21.15.46:0/372932493 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820db67640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82080af300 0x7f820813ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820d366640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f8208070940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 -- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82080af300 msgr2=0x7f820813ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82080af300 0x7f820813ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 -- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 msgr2=0x7f8208070940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f8208070940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 -- 172.21.15.46:0/372932493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8204009420 con 0x7f82081451c0 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820d366640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f8208070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820e368640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82081451c0 0x7f82081366e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f820000efc0 tx=0x7f820000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f81fe7fc640 1 -- 172.21.15.46:0/372932493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000cd20 con 0x7f82081451c0 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f81fe7fc640 1 -- 172.21.15.46:0/372932493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820000f040 con 0x7f82081451c0 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8208071000 con 0x7f82081451c0 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f81fe7fc640 1 -- 172.21.15.46:0/372932493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820000cea0 con 0x7f82081451c0 2023-12-16T21:43:21.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82081dd7c0 con 0x7f82081451c0 2023-12-16T21:43:21.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82081363c0 con 0x7f82081451c0 2023-12-16T21:43:21.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f81fe7fc640 1 -- 172.21.15.46:0/372932493 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8200003be0 con 0x7f82081451c0 2023-12-16T21:43:21.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f81fe7fc640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81ec066470 0x7f81ec068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.127+0000 7f81fe7fc640 1 -- 172.21.15.46:0/372932493 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8200084270 con 0x7f82081451c0 2023-12-16T21:43:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.131+0000 7f820db67640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81ec066470 0x7f81ec068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.131+0000 7f820db67640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81ec066470 0x7f81ec068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f82040af000 tx=0x7f82040a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:21.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.135+0000 7f81fe7fc640 1 -- 172.21.15.46:0/372932493 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8200051780 con 0x7f82081451c0 2023-12-16T21:43:21.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.323+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8208068960 con 0x7f82081451c0 2023-12-16T21:43:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.347+0000 7f81fe7fc640 1 -- 172.21.15.46:0/372932493 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8200054da0 con 0x7f82081451c0 2023-12-16T21:43:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81ec066470 msgr2=0x7f81ec068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81ec066470 0x7f81ec068930 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f82040af000 tx=0x7f82040a7000 comp rx=0 tx=0).stop 2023-12-16T21:43:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82081451c0 msgr2=0x7f82081366e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82081451c0 0x7f82081366e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f820000efc0 tx=0x7f820000c490 comp rx=0 tx=0).stop 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 shutdown_connections 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f81ec066470 0x7f81ec068930 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82081451c0 0x7f82081366e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f82080b1eb0 0x7f8208070940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 --2- 172.21.15.46:0/372932493 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f82080af300 0x7f820813ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 >> 172.21.15.46:0/372932493 conn(0x7f82080b4900 msgr2=0x7f82080b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 shutdown_connections 2023-12-16T21:43:21.356 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.351+0000 7f820fdcb640 1 -- 172.21.15.46:0/372932493 wait complete. 2023-12-16T21:43:21.372 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-16T21:43:21.372 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-16T21:43:21.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa7789a4640 1 Processor -- start 2023-12-16T21:43:21.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa7789a4640 1 -- start start 2023-12-16T21:43:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa7789a4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa77406ce70 0x7fa774070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa7789a4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa7740afc50 0x7fa77413d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa7789a4640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7740705f0 con 0x7fa77413d6a0 2023-12-16T21:43:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa7789a4640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa774071f60 con 0x7fa7740afc50 2023-12-16T21:43:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa7789a4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b5180 con 0x7fa77406ce70 2023-12-16T21:43:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa772d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fa77413d6a0 0x7fa7741433e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42456/0 (socket says 172.21.15.46:42456) 2023-12-16T21:43:21.494 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa772d9d640 1 -- 172.21.15.46:0/539309768 learned_addr learned my addr 172.21.15.46:0/539309768 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa771d9b640 1 --2- 172.21.15.46:0/539309768 >> v2:172.21.15.161:3300/0 conn(0x7fa7740afc50 0x7fa77413d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77259c640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa77406ce70 0x7fa774070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848253719 0 0) 0x7fa7740705f0 con 0x7fa77413d6a0 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c003530 con 0x7fa77413d6a0 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4074641893 0 0) 0x7fa75c003530 con 0x7fa77413d6a0 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa77406ce70 msgr2=0x7fa774070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa77406ce70 0x7fa774070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 >> v2:172.21.15.161:3300/0 conn(0x7fa7740afc50 msgr2=0x7fa77413d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 --2- 172.21.15.46:0/539309768 >> v2:172.21.15.161:3300/0 conn(0x7fa7740afc50 0x7fa77413d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77406d250 con 0x7fa77413d6a0 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fa76c001a00 con 0x7fa77413d6a0 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 >> v1:172.21.15.46:6789/0 conn(0x7fa77413d6a0 legacy=0x7fa7741433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa75c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c0047d0 0x7fa75c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa77406ce70 0x7fa75c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774071f60 con 0x7fa75c003980 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b5180 con 0x7fa75c0047d0 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c00b990 con 0x7fa77406ce70 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77259c640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa75c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa771d9b640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c0047d0 0x7fa75c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77259c640 1 -- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa77406ce70 msgr2=0x7fa75c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:21.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77259c640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa77406ce70 0x7fa75c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77259c640 1 -- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c0047d0 msgr2=0x7fa75c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77259c640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa75c0047d0 0x7fa75c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.491+0000 7fa77259c640 1 -- 172.21.15.46:0/539309768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c003530 con 0x7fa75c003980 2023-12-16T21:43:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa75c003da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa764009b10 tx=0x7fa76401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e444c4632d28e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640a6510 con 0x7fa75c003980 2023-12-16T21:43:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7640a6670 con 0x7fa75c003980 2023-12-16T21:43:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77159a640 1 -- 172.21.15.46:0/539309768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640a6a20 con 0x7fa75c003980 2023-12-16T21:43:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- 172.21.15.46:0/539309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 msgr2=0x7fa75c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 --2- 172.21.15.46:0/539309768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa75c003da0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa764009b10 tx=0x7fa76401ca90 comp rx=0 tx=0).stop 2023-12-16T21:43:21.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa772d9d640 1 -- 172.21.15.46:0/539309768 reap_dead start 2023-12-16T21:43:21.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- 172.21.15.46:0/539309768 shutdown_connections 2023-12-16T21:43:21.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- 172.21.15.46:0/539309768 >> 172.21.15.46:0/539309768 conn(0x7fa7740b90b0 msgr2=0x7fa7740b94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:21.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- 172.21.15.46:0/539309768 shutdown_connections 2023-12-16T21:43:21.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- 172.21.15.46:0/539309768 wait complete. 2023-12-16T21:43:21.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 Processor -- start 2023-12-16T21:43:21.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- start start 2023-12-16T21:43:21.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa7741415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa774141ae0 0x7fa7741e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7741e7090 0x7fa7741e7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774144380 con 0x7fa75c003980 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774143800 con 0x7fa7741e7090 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa7789a4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa774141f40 con 0x7fa774141ae0 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa7741415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa7741415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53742/0 (socket says 172.21.15.46:53742) 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 -- 172.21.15.46:0/3127843004 learned_addr learned my addr 172.21.15.46:0/3127843004 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa772d9d640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7741e7090 0x7fa7741e7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa771d9b640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa774141ae0 0x7fa7741e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 -- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa774141ae0 msgr2=0x7fa7741e6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa774141ae0 0x7fa7741e6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 -- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7741e7090 msgr2=0x7fa7741e7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:21.502 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7741e7090 0x7fa7741e7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.495+0000 7fa77259c640 1 -- 172.21.15.46:0/3127843004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa774143520 con 0x7fa75c003980 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa77259c640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa7741415a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7640b0040 tx=0x7fa75c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa762ffd640 1 -- 172.21.15.46:0/3127843004 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7640a69e0 con 0x7fa75c003980 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa762ffd640 1 -- 172.21.15.46:0/3127843004 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7640a6b60 con 0x7fa75c003980 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7740b2c00 con 0x7fa75c003980 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa762ffd640 1 -- 172.21.15.46:0/3127843004 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764006040 con 0x7fa75c003980 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7740b30f0 con 0x7fa75c003980 2023-12-16T21:43:21.503 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774067f60 con 0x7fa75c003980 2023-12-16T21:43:21.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa762ffd640 1 -- 172.21.15.46:0/3127843004 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fa7640061a0 con 0x7fa75c003980 2023-12-16T21:43:21.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa762ffd640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa754066470 0x7fa754068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:21.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa762ffd640 1 -- 172.21.15.46:0/3127843004 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7640aa070 con 0x7fa75c003980 2023-12-16T21:43:21.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa771d9b640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa754066470 0x7fa754068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:21.504 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.499+0000 7fa771d9b640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa754066470 0x7fa754068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa774067740 tx=0x7fa768008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:21.506 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.503+0000 7fa762ffd640 1 -- 172.21.15.46:0/3127843004 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fa7640ec450 con 0x7fa75c003980 2023-12-16T21:43:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:21.691+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa774068960 con 0x7fa75c003980 2023-12-16T21:43:22.354 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.351+0000 7fa762ffd640 1 -- 172.21.15.46:0/3127843004 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa7640efa70 con 0x7fa75c003980 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa754066470 msgr2=0x7fa754068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa754066470 0x7fa754068930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa774067740 tx=0x7fa768008040 comp rx=0 tx=0).stop 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 msgr2=0x7fa7741415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa7741415a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa7640b0040 tx=0x7fa75c005800 comp rx=0 tx=0).stop 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 shutdown_connections 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa754066470 0x7fa754068930 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa7741e7090 0x7fa7741e7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa774141ae0 0x7fa7741e6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 --2- 172.21.15.46:0/3127843004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa75c003980 0x7fa7741415a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.359 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 >> 172.21.15.46:0/3127843004 conn(0x7fa7740b90b0 msgr2=0x7fa7740b5e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 shutdown_connections 2023-12-16T21:43:22.360 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.355+0000 7fa7789a4640 1 -- 172.21.15.46:0/3127843004 wait complete. 2023-12-16T21:43:22.381 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-16T21:43:22.381 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f317d27f640 1 Processor -- start 2023-12-16T21:43:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f317d27f640 1 -- start start 2023-12-16T21:43:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f317d27f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f31780b3fd0 0x7f317813e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f317d27f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31780b0340 0x7f317813e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f317d27f640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31780af2e0 con 0x7f317806c920 2023-12-16T21:43:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f317d27f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31780b1f50 con 0x7f31780b3fd0 2023-12-16T21:43:22.510 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f317d27f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317813eda0 con 0x7f31780b0340 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31767fc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f31780b3fd0 0x7f317813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31777fe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31780b0340 0x7f317813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31767fc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f31780b3fd0 0x7f317813e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:59892/0 (socket says 172.21.15.46:59892) 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31767fc640 1 -- 172.21.15.46:0/3988369999 learned_addr learned my addr 172.21.15.46:0/3988369999 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31777fe640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31780b0340 0x7f317813e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:48150/0 (socket says 172.21.15.46:48150) 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31777fe640 1 -- 172.21.15.46:0/3988369999 >> v2:172.21.15.161:3300/0 conn(0x7f31780b3fd0 msgr2=0x7f317813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31777fe640 1 --2- 172.21.15.46:0/3988369999 >> v2:172.21.15.161:3300/0 conn(0x7f31780b3fd0 0x7f317813e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31777fe640 1 -- 172.21.15.46:0/3988369999 >> v1:172.21.15.161:6789/0 conn(0x7f317806c920 legacy=0x7f31780afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31777fe640 1 -- 172.21.15.46:0/3988369999 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317806cd00 con 0x7f31780b0340 2023-12-16T21:43:22.511 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.507+0000 7f31767fc640 1 --2- 172.21.15.46:0/3988369999 >> v2:172.21.15.161:3300/0 conn(0x7f31780b3fd0 0x7f317813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:22.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31777fe640 1 --2- 172.21.15.46:0/3988369999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31780b0340 0x7f317813e860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3170009d90 tx=0x7f317001c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34dda9a4b615756 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f3175ffb640 1 -- 172.21.15.46:0/3988369999 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700a5780 con 0x7f31780b0340 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f3175ffb640 1 -- 172.21.15.46:0/3988369999 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31700a58e0 con 0x7f31780b0340 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f3175ffb640 1 -- 172.21.15.46:0/3988369999 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31700a5c70 con 0x7f31780b0340 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- 172.21.15.46:0/3988369999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31780b0340 msgr2=0x7f317813e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 --2- 172.21.15.46:0/3988369999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31780b0340 0x7f317813e860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3170009d90 tx=0x7f317001c8c0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- 172.21.15.46:0/3988369999 shutdown_connections 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 --2- 172.21.15.46:0/3988369999 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31780b0340 0x7f317813e860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 --2- 172.21.15.46:0/3988369999 >> v2:172.21.15.161:3300/0 conn(0x7f31780b3fd0 0x7f317813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- 172.21.15.46:0/3988369999 >> 172.21.15.46:0/3988369999 conn(0x7f31780b48e0 msgr2=0x7f31780b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- 172.21.15.46:0/3988369999 shutdown_connections 2023-12-16T21:43:22.515 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- 172.21.15.46:0/3988369999 wait complete. 2023-12-16T21:43:22.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 Processor -- start 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- start start 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317806c920 0x7f3178140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31781e29b0 0x7f31781df3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31781df900 0x7f31780708d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31781e35e0 con 0x7f31781e29b0 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3178141150 con 0x7f317806c920 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f317d27f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31781dfd80 con 0x7f31781df900 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31781e29b0 0x7f31781df3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31781e29b0 0x7f31781df3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53752/0 (socket says 172.21.15.46:53752) 2023-12-16T21:43:22.518 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 -- 172.21.15.46:0/177289332 learned_addr learned my addr 172.21.15.46:0/177289332 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f3176ffd640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317806c920 0x7f3178140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31777fe640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31781df900 0x7f31780708d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 -- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31781df900 msgr2=0x7f31780708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31781df900 0x7f31780708d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 -- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317806c920 msgr2=0x7f3178140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317806c920 0x7f3178140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.511+0000 7f31767fc640 1 -- 172.21.15.46:0/177289332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31680094d0 con 0x7f31781e29b0 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f31767fc640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31781e29b0 0x7f31781df3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f316800e990 tx=0x7f316800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f315f7fe640 1 -- 172.21.15.46:0/177289332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316800c9e0 con 0x7f31781e29b0 2023-12-16T21:43:22.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f315f7fe640 1 -- 172.21.15.46:0/177289332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f316800cb40 con 0x7f31781e29b0 2023-12-16T21:43:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3170009a70 con 0x7f31781e29b0 2023-12-16T21:43:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f315f7fe640 1 -- 172.21.15.46:0/177289332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f316800ceb0 con 0x7f31781e29b0 2023-12-16T21:43:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31781e3200 con 0x7f31781e29b0 2023-12-16T21:43:22.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31780679e0 con 0x7f31781e29b0 2023-12-16T21:43:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f315f7fe640 1 -- 172.21.15.46:0/177289332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f3168013470 con 0x7f31781e29b0 2023-12-16T21:43:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f315f7fe640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31640664a0 0x7f3164068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f315f7fe640 1 -- 172.21.15.46:0/177289332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31680846c0 con 0x7f31781e29b0 2023-12-16T21:43:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f3176ffd640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31640664a0 0x7f3164068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.515+0000 7f3176ffd640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31640664a0 0x7f3164068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f316c001810 tx=0x7f316c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:22.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.519+0000 7f315f7fe640 1 -- 172.21.15.46:0/177289332 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f3168051d60 con 0x7f31781e29b0 2023-12-16T21:43:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.699+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31781df1e0 con 0x7f31781e29b0 2023-12-16T21:43:22.704 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.699+0000 7f315f7fe640 1 -- 172.21.15.46:0/177289332 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3359 (secure 0 0 0) 0x7f3168021d30 con 0x7f31781e29b0 2023-12-16T21:43:22.704 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:22.705 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3332084711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3332084711},{"type":"v1","addr":"172.21.15.161:6835","nonce":3332084711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:43:19.618214+0000","modified":"2023-12-16T21:43:21.693210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/3724256427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3724256427},{"type":"v1","addr":"172.21.15.46:6833","nonce":3724256427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-16T21:43:22.705 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-16T21:43:22.706 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31640664a0 msgr2=0x7f3164068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31640664a0 0x7f3164068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f316c001810 tx=0x7f316c009040 comp rx=0 tx=0).stop 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31781e29b0 msgr2=0x7f31781df3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31781e29b0 0x7f31781df3c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f316800e990 tx=0x7f316800ee50 comp rx=0 tx=0).stop 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 shutdown_connections 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f31640664a0 0x7f3164068960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f31781df900 0x7f31780708d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31781e29b0 0x7f31781df3c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 --2- 172.21.15.46:0/177289332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f317806c920 0x7f3178140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 >> 172.21.15.46:0/177289332 conn(0x7f31780b48e0 msgr2=0x7f3178137b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 shutdown_connections 2023-12-16T21:43:22.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.703+0000 7f317d27f640 1 -- 172.21.15.46:0/177289332 wait complete. 2023-12-16T21:43:22.722 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.839+0000 7f484832e640 1 Processor -- start 2023-12-16T21:43:22.844 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.839+0000 7f484832e640 1 -- start start 2023-12-16T21:43:22.845 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f484006c920 0x7f48400adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48400ae840 0x7f48400b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f48400b0be0 0x7f48400b1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48400b4770 con 0x7f484006c920 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48400adad0 con 0x7f48400b0be0 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48400b2d20 con 0x7f48400ae840 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f484006c920 0x7f48400adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f484006c920 0x7f48400adf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53768/0 (socket says 172.21.15.46:53768) 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 -- 172.21.15.46:0/3450788787 learned_addr learned my addr 172.21.15.46:0/3450788787 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48468cb640 1 --2- 172.21.15.46:0/3450788787 >> v2:172.21.15.161:3300/0 conn(0x7f48400b0be0 0x7f48400b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48458c9640 1 --2- 172.21.15.46:0/3450788787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48400ae840 0x7f48400b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 -- 172.21.15.46:0/3450788787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48400ae840 msgr2=0x7f48400b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 --2- 172.21.15.46:0/3450788787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48400ae840 0x7f48400b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 -- 172.21.15.46:0/3450788787 >> v2:172.21.15.161:3300/0 conn(0x7f48400b0be0 msgr2=0x7f48400b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:22.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 --2- 172.21.15.46:0/3450788787 >> v2:172.21.15.161:3300/0 conn(0x7f48400b0be0 0x7f48400b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 -- 172.21.15.46:0/3450788787 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484006cd00 con 0x7f484006c920 2023-12-16T21:43:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48460ca640 1 --2- 172.21.15.46:0/3450788787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484006c920 0x7f48400adf60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f48400b3a20 tx=0x7f483c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffe14a52405cc88 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:22.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48450c8640 1 -- 172.21.15.46:0/3450788787 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c0ab600 con 0x7f484006c920 2023-12-16T21:43:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48450c8640 1 -- 172.21.15.46:0/3450788787 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c0a7050 con 0x7f484006c920 2023-12-16T21:43:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f48450c8640 1 -- 172.21.15.46:0/3450788787 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c0ab980 con 0x7f484006c920 2023-12-16T21:43:22.848 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 -- 172.21.15.46:0/3450788787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484006c920 msgr2=0x7f48400adf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 --2- 172.21.15.46:0/3450788787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484006c920 0x7f48400adf60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f48400b3a20 tx=0x7f483c01c3e0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 -- 172.21.15.46:0/3450788787 shutdown_connections 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 --2- 172.21.15.46:0/3450788787 >> v2:172.21.15.161:3300/0 conn(0x7f48400b0be0 0x7f48400b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 --2- 172.21.15.46:0/3450788787 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f48400ae840 0x7f48400b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 --2- 172.21.15.46:0/3450788787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f484006c920 0x7f48400adf60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 -- 172.21.15.46:0/3450788787 >> 172.21.15.46:0/3450788787 conn(0x7f48400b48e0 msgr2=0x7f48400b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.843+0000 7f484832e640 1 -- 172.21.15.46:0/3450788787 shutdown_connections 2023-12-16T21:43:22.849 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 -- 172.21.15.46:0/3450788787 wait complete. 2023-12-16T21:43:22.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 Processor -- start 2023-12-16T21:43:22.850 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 -- start start 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484006c920 0x7f48400712a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48400ae840 0x7f484014b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48400b0be0 0x7f48401e29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484014be30 con 0x7f48400b0be0 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48401e2250 con 0x7f48400ae840 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4840143c40 con 0x7f484006c920 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48400b0be0 0x7f48401e29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.852 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48400b0be0 0x7f48401e29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53780/0 (socket says 172.21.15.46:53780) 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 -- 172.21.15.46:0/4211917504 learned_addr learned my addr 172.21.15.46:0/4211917504 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48460ca640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484006c920 0x7f48400712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48458c9640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48400ae840 0x7f484014b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 -- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484006c920 msgr2=0x7f48400712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484006c920 0x7f48400712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 -- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48400ae840 msgr2=0x7f484014b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48400ae840 0x7f484014b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 -- 172.21.15.46:0/4211917504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0094d0 con 0x7f48400b0be0 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48460ca640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484006c920 0x7f48400712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f48468cb640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48400b0be0 0x7f48401e29c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4838007e60 tx=0x7f483800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:22.853 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f4836ffd640 1 -- 172.21.15.46:0/4211917504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838014420 con 0x7f48400b0be0 2023-12-16T21:43:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f4836ffd640 1 -- 172.21.15.46:0/4211917504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48380145a0 con 0x7f48400b0be0 2023-12-16T21:43:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484014c070 con 0x7f48400b0be0 2023-12-16T21:43:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f4836ffd640 1 -- 172.21.15.46:0/4211917504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838014950 con 0x7f48400b0be0 2023-12-16T21:43:22.854 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.847+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4840143010 con 0x7f48400b0be0 2023-12-16T21:43:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.851+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48400679e0 con 0x7f48400b0be0 2023-12-16T21:43:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.851+0000 7f4836ffd640 1 -- 172.21.15.46:0/4211917504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f4838014ab0 con 0x7f48400b0be0 2023-12-16T21:43:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.851+0000 7f4836ffd640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4824066470 0x7f4824068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.851+0000 7f4836ffd640 1 -- 172.21.15.46:0/4211917504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4838084690 con 0x7f48400b0be0 2023-12-16T21:43:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.851+0000 7f48460ca640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4824066470 0x7f4824068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:22.855 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.851+0000 7f48460ca640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4824066470 0x7f4824068930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f483c0ad000 tx=0x7f483c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:22.858 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:22.855+0000 7f4836ffd640 1 -- 172.21.15.46:0/4211917504 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f4838051ba0 con 0x7f48400b0be0 2023-12-16T21:43:23.008 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.003+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48400688f0 con 0x7f48400b0be0 2023-12-16T21:43:23.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.007+0000 7f4836ffd640 1 -- 172.21.15.46:0/4211917504 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f48380551c0 con 0x7f48400b0be0 2023-12-16T21:43:23.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:23.011 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:21.267009+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:43:17.887506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-16T21:43:18.612695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.007+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4824066470 msgr2=0x7f4824068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.007+0000 7f484832e640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4824066470 0x7f4824068930 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f483c0ad000 tx=0x7f483c008040 comp rx=0 tx=0).stop 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.007+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48400b0be0 msgr2=0x7f48401e29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.007+0000 7f484832e640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48400b0be0 0x7f48401e29c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4838007e60 tx=0x7f483800e4b0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 shutdown_connections 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4824066470 0x7f4824068930 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48400b0be0 0x7f48401e29c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48400ae840 0x7f484014b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 --2- 172.21.15.46:0/4211917504 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f484006c920 0x7f48400712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 >> 172.21.15.46:0/4211917504 conn(0x7f48400b48e0 msgr2=0x7f48400b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:23.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 shutdown_connections 2023-12-16T21:43:23.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.011+0000 7f484832e640 1 -- 172.21.15.46:0/4211917504 wait complete. 2023-12-16T21:43:23.029 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-16T21:43:23.030 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc12b90640 1 Processor -- start 2023-12-16T21:43:23.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc12b90640 1 -- start start 2023-12-16T21:43:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc12b90640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c0bc800 0x7fbc0c0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc12b90640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0c140ab0 con 0x7fbc0c13ff50 2023-12-16T21:43:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc12b90640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0c0bcbe0 con 0x7fbc0c149c80 2023-12-16T21:43:23.142 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc12b90640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c0b38f0 con 0x7fbc0c0bc800 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fbc0c13ff50 0x7fbc0c1403a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42458/0 (socket says 172.21.15.46:42458) 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 -- 172.21.15.46:0/3169888099 learned_addr learned my addr 172.21.15.46:0/3169888099 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1092c640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c0bc800 0x7fbc0c0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101838707 0 0) 0x7fbc040014a0 con 0x7fbc0c13ff50 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbf4003530 con 0x7fbc0c13ff50 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1882107259 0 0) 0x7fbbf4003530 con 0x7fbc0c13ff50 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c0bc800 msgr2=0x7fbc0c0b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c0bc800 0x7fbc0c0b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 >> v1:172.21.15.161:6789/0 conn(0x7fbc0c149c80 legacy=0x7fbc0c14a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0b3bd0 con 0x7fbc0c13ff50 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fbc04001b90 con 0x7fbc0c13ff50 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 >> v1:172.21.15.46:6789/0 conn(0x7fbc0c13ff50 legacy=0x7fbc0c1403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.144 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf40039b0 0x7fbbf4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbf4004360 0x7fbbf4004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbbf400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c0b38f0 con 0x7fbbf4004fe0 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf400bf00 con 0x7fbbf40039b0 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf400c0c0 con 0x7fbbf4004360 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1092c640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c0bc800 0x7fbc0c0b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbbf400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc0bfff640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbf4004360 0x7fbbf4004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1092c640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf40039b0 0x7fbbf4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 -- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbf4004360 msgr2=0x7fbbf4004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbbf4004360 0x7fbbf4004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 -- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf40039b0 msgr2=0x7fbbf4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf40039b0 0x7fbbf4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.145 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1112d640 1 -- 172.21.15.46:0/3169888099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf4003530 con 0x7fbbf4004fe0 2023-12-16T21:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.139+0000 7fbc1092c640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbbf40039b0 0x7fbbf4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc1112d640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbbf400b990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400a830 tx=0x7fbc0401fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a4efff507d7a50 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc040aa6b0 con 0x7fbbf4004fe0 2023-12-16T21:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc040aa810 con 0x7fbbf4004fe0 2023-12-16T21:43:23.146 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc0b7fe640 1 -- 172.21.15.46:0/3169888099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc040aabc0 con 0x7fbbf4004fe0 2023-12-16T21:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc12b90640 1 -- 172.21.15.46:0/3169888099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 msgr2=0x7fbbf400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc12b90640 1 --2- 172.21.15.46:0/3169888099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbbf400b990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400a830 tx=0x7fbc0401fa80 comp rx=0 tx=0).stop 2023-12-16T21:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc1112d640 1 -- 172.21.15.46:0/3169888099 reap_dead start 2023-12-16T21:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc12b90640 1 -- 172.21.15.46:0/3169888099 shutdown_connections 2023-12-16T21:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc12b90640 1 -- 172.21.15.46:0/3169888099 >> 172.21.15.46:0/3169888099 conn(0x7fbc0c0722b0 msgr2=0x7fbc0c0bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc12b90640 1 -- 172.21.15.46:0/3169888099 shutdown_connections 2023-12-16T21:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc12b90640 1 -- 172.21.15.46:0/3169888099 wait complete. 2023-12-16T21:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.143+0000 7fbc12b90640 1 Processor -- start 2023-12-16T21:43:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 -- start start 2023-12-16T21:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbc0c13acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c1dbbd0 0x7fbc0c1dc050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc0c1df140 0x7fbc0c1df5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c0b2840 con 0x7fbbf4004fe0 2023-12-16T21:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c136ec0 con 0x7fbc0c1df140 2023-12-16T21:43:23.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0c1463d0 con 0x7fbc0c1dbbd0 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbc0c13acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbc0c13acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53800/0 (socket says 172.21.15.46:53800) 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 -- 172.21.15.46:0/2266535503 learned_addr learned my addr 172.21.15.46:0/2266535503 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1112d640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc0c1df140 0x7fbc0c1df5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc0bfff640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c1dbbd0 0x7fbc0c1dc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 -- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c1dbbd0 msgr2=0x7fbc0c1dc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c1dbbd0 0x7fbc0c1dc050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 -- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc0c1df140 msgr2=0x7fbc0c1df5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc0c1df140 0x7fbc0c1df5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 -- 172.21.15.46:0/2266535503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c1373f0 con 0x7fbbf4004fe0 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1112d640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc0c1df140 0x7fbc0c1df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc1092c640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbc0c13acc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc010900 tx=0x7fbbfc010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc097fa640 1 -- 172.21.15.46:0/2266535503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc00eb60 con 0x7fbbf4004fe0 2023-12-16T21:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc097fa640 1 -- 172.21.15.46:0/2266535503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbfc00ecc0 con 0x7fbbf4004fe0 2023-12-16T21:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc097fa640 1 -- 172.21.15.46:0/2266535503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc01c400 con 0x7fbbf4004fe0 2023-12-16T21:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc0c135f70 con 0x7fbbf4004fe0 2023-12-16T21:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc0c1dc590 con 0x7fbbf4004fe0 2023-12-16T21:43:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc0c067de0 con 0x7fbbf4004fe0 2023-12-16T21:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc097fa640 1 -- 172.21.15.46:0/2266535503 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fbbfc01c560 con 0x7fbbf4004fe0 2023-12-16T21:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc097fa640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbbf80664a0 0x7fbbf8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.147+0000 7fbc097fa640 1 -- 172.21.15.46:0/2266535503 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbbfc0843d0 con 0x7fbbf4004fe0 2023-12-16T21:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.151+0000 7fbc0bfff640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbbf80664a0 0x7fbbf8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.151+0000 7fbc0bfff640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbbf80664a0 0x7fbbf8068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0670f0 tx=0x7fbc00009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.155+0000 7fbc097fa640 1 -- 172.21.15.46:0/2266535503 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fbbfc051990 con 0x7fbbf4004fe0 2023-12-16T21:43:23.312 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.307+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbc0c0b85e0 con 0x7fbbf80664a0 2023-12-16T21:43:23.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.327+0000 7fbc097fa640 1 -- 172.21.15.46:0/2266535503 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249866 (secure 0 0 0) 0x7fbc0000d050 con 0x7fbbf80664a0 2023-12-16T21:43:23.333 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:23.335 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbbf80664a0 msgr2=0x7fbbf8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbbf80664a0 0x7fbbf8068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0670f0 tx=0x7fbc00009040 comp rx=0 tx=0).stop 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 msgr2=0x7fbc0c13acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbc0c13acc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc010900 tx=0x7fbbfc010dc0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 shutdown_connections 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fbbf80664a0 0x7fbbf8068960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fbc0c1df140 0x7fbc0c1df5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fbc0c1dbbd0 0x7fbc0c1dc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 --2- 172.21.15.46:0/2266535503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbbf4004fe0 0x7fbc0c13acc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 >> 172.21.15.46:0/2266535503 conn(0x7fbc0c0722b0 msgr2=0x7fbc0c0b9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 shutdown_connections 2023-12-16T21:43:23.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.331+0000 7fbc12b90640 1 -- 172.21.15.46:0/2266535503 wait complete. 2023-12-16T21:43:23.352 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2023-12-16T21:43:22.364480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236232,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738488,"statfs":{"total":773094113280,"available":772852211712,"internally_reserved":0,"allocated":3481600,"data_stored":2008736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":77,"apply_latency_ms":77,"commit_latency_ns":77000000,"apply_latency_ns":77000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.121185"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-12-16T21:43:14.028059+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:14.028059+0000","last_peered":"2023-12-16T21:43:14.028059+0000","last_clean":"2023-12-16T21:43:14.028059+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:14.028059+0000","last_undegraded":"2023-12-16T21:43:14.028059+0000","last_fullsized":"2023-12-16T21:43:14.028059+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":693868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":693868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:23.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:23.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.511+0000 7f7f181af640 1 Processor -- start 2023-12-16T21:43:23.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.511+0000 7f7f181af640 1 -- start start 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f181af640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7f100b12f0 0x7f7f100adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f181af640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f100adad0 con 0x7f7f1006c920 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f181af640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1006cd00 con 0x7f7f100ae840 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f181af640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f100b4770 con 0x7f7f100b12f0 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f15f4b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f7f1006c920 0x7f7f100b27e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42462/0 (socket says 172.21.15.46:42462) 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f15f4b640 1 -- 172.21.15.46:0/1493462895 learned_addr learned my addr 172.21.15.46:0/1493462895 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1674c640 1 --2- 172.21.15.46:0/1493462895 >> v2:172.21.15.161:3300/0 conn(0x7f7f100b12f0 0x7f7f100adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388557403 0 0) 0x7f7f100adad0 con 0x7f7f1006c920 2023-12-16T21:43:23.519 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ef8003530 con 0x7f7f1006c920 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3404776963 0 0) 0x7f7ef8003530 con 0x7f7f1006c920 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 >> v2:172.21.15.161:3300/0 conn(0x7f7f100b12f0 msgr2=0x7f7f100adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 --2- 172.21.15.46:0/1493462895 >> v2:172.21.15.161:3300/0 conn(0x7f7f100b12f0 0x7f7f100adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 >> v1:172.21.15.161:6789/0 conn(0x7f7f100ae840 legacy=0x7f7f100b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100b4440 con 0x7f7f1006c920 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f7f00001680 con 0x7f7f1006c920 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 >> v1:172.21.15.46:6789/0 conn(0x7f7f1006c920 legacy=0x7f7f100b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef8003980 0x7f7ef8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ef80047d0 0x7f7ef8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef80052b0 0x7f7ef800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f100b4770 con 0x7f7ef80052b0 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef800c180 con 0x7f7ef8003980 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef800c340 con 0x7f7ef80047d0 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1674c640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef80052b0 0x7f7ef800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1574a640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ef80047d0 0x7f7ef8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f15f4b640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef8003980 0x7f7ef8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1674c640 1 -- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ef80047d0 msgr2=0x7f7ef8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1674c640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ef80047d0 0x7f7ef8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1674c640 1 -- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef8003980 msgr2=0x7f7ef8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1674c640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef8003980 0x7f7ef8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.515+0000 7f7f1674c640 1 -- 172.21.15.46:0/1493462895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef8003530 con 0x7f7ef80052b0 2023-12-16T21:43:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1674c640 1 -- 172.21.15.46:0/1493462895 reap_dead start 2023-12-16T21:43:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1674c640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef80052b0 0x7f7ef800b990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009ab0 tx=0x7f7f0801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=347368f1b810e50e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08004020 con 0x7f7ef80052b0 2023-12-16T21:43:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f08004180 con 0x7f7ef80052b0 2023-12-16T21:43:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f14f49640 1 -- 172.21.15.46:0/1493462895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080a5660 con 0x7f7ef80052b0 2023-12-16T21:43:23.522 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- 172.21.15.46:0/1493462895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef80052b0 msgr2=0x7f7ef800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef80052b0 0x7f7ef800b990 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009ab0 tx=0x7f7f0801ca70 comp rx=0 tx=0).stop 2023-12-16T21:43:23.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- 172.21.15.46:0/1493462895 shutdown_connections 2023-12-16T21:43:23.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 --2- 172.21.15.46:0/1493462895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ef80052b0 0x7f7ef800b990 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- 172.21.15.46:0/1493462895 >> 172.21.15.46:0/1493462895 conn(0x7f7f100b48e0 msgr2=0x7f7f100b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:23.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- 172.21.15.46:0/1493462895 shutdown_connections 2023-12-16T21:43:23.523 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- 172.21.15.46:0/1493462895 wait complete. 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 Processor -- start 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- start start 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef80052b0 0x7f7f1013b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f1013bde0 0x7f7f1006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f1006f630 0x7f7f100704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f101df900 con 0x7f7f1013bde0 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1006fac0 con 0x7f7ef80052b0 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f101df2b0 con 0x7f7f1006f630 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f1013bde0 0x7f7f1006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f1013bde0 0x7f7f1006f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53810/0 (socket says 172.21.15.46:53810) 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 -- 172.21.15.46:0/2592435294 learned_addr learned my addr 172.21.15.46:0/2592435294 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1674c640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f1006f630 0x7f7f100704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.525 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f15f4b640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef80052b0 0x7f7f1013b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 -- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f1006f630 msgr2=0x7f7f100704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f1006f630 0x7f7f100704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 -- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef80052b0 msgr2=0x7f7f1013b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef80052b0 0x7f7f1013b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 -- 172.21.15.46:0/2592435294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f101df590 con 0x7f7f1013bde0 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f15f4b640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef80052b0 0x7f7f1013b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f1574a640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f1013bde0 0x7f7f1006f0f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400e7f0 tx=0x7f7f0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f0e7fc640 1 -- 172.21.15.46:0/2592435294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f04003b90 con 0x7f7f1013bde0 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f0e7fc640 1 -- 172.21.15.46:0/2592435294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f04003cf0 con 0x7f7f1013bde0 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f10071730 con 0x7f7f1013bde0 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f0e7fc640 1 -- 172.21.15.46:0/2592435294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f040052d0 con 0x7f7f1013bde0 2023-12-16T21:43:23.526 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.519+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f10147a90 con 0x7f7f1013bde0 2023-12-16T21:43:23.527 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.523+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f100679e0 con 0x7f7f1013bde0 2023-12-16T21:43:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.523+0000 7f7f0e7fc640 1 -- 172.21.15.46:0/2592435294 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7f04005430 con 0x7f7f1013bde0 2023-12-16T21:43:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.523+0000 7f7f0e7fc640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7efc0664a0 0x7f7efc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.523+0000 7f7f0e7fc640 1 -- 172.21.15.46:0/2592435294 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f04083eb0 con 0x7f7f1013bde0 2023-12-16T21:43:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.523+0000 7f7f15f4b640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7efc0664a0 0x7f7efc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.528 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.523+0000 7f7f15f4b640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7efc0664a0 0x7f7efc068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f00001ec0 tx=0x7f7f00000ff0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.530 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.527+0000 7f7f0e7fc640 1 -- 172.21.15.46:0/2592435294 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f7f040513c0 con 0x7f7f1013bde0 2023-12-16T21:43:23.676 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.675+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f101df420 con 0x7f7efc0664a0 2023-12-16T21:43:23.695 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.691+0000 7f7f0e7fc640 1 -- 172.21.15.46:0/2592435294 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249866 (secure 0 0 0) 0x7f7f101df420 con 0x7f7efc0664a0 2023-12-16T21:43:23.697 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:23.699 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7efc0664a0 msgr2=0x7f7efc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7efc0664a0 0x7f7efc068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f00001ec0 tx=0x7f7f00000ff0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f1013bde0 msgr2=0x7f7f1006f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f1013bde0 0x7f7f1006f0f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7f0400e7f0 tx=0x7f7f0400ecb0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 shutdown_connections 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7efc0664a0 0x7f7efc068960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7f1006f630 0x7f7f100704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7f1013bde0 0x7f7f1006f0f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 --2- 172.21.15.46:0/2592435294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ef80052b0 0x7f7f1013b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 >> 172.21.15.46:0/2592435294 conn(0x7f7f100b48e0 msgr2=0x7f7f1013f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 shutdown_connections 2023-12-16T21:43:23.700 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.695+0000 7f7f181af640 1 -- 172.21.15.46:0/2592435294 wait complete. 2023-12-16T21:43:23.714 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2023-12-16T21:43:22.364480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236232,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738488,"statfs":{"total":773094113280,"available":772852211712,"internally_reserved":0,"allocated":3481600,"data_stored":2008736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":77,"apply_latency_ms":77,"commit_latency_ns":77000000,"apply_latency_ns":77000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.121185"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-12-16T21:43:14.028059+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:14.028059+0000","last_peered":"2023-12-16T21:43:14.028059+0000","last_clean":"2023-12-16T21:43:14.028059+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:14.028059+0000","last_undegraded":"2023-12-16T21:43:14.028059+0000","last_fullsized":"2023-12-16T21:43:14.028059+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":693868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":693868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:23.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 Processor -- start 2023-12-16T21:43:23.877 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 -- start start 2023-12-16T21:43:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21540bc800 0x7f21540b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2154149c80 0x7f215414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f215413ff50 0x7f21541403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215414a620 con 0x7f2154149c80 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21540bcbe0 con 0x7f215413ff50 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f215b35f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540b38f0 con 0x7f21540bc800 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2154149c80 0x7f215414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f2154149c80 0x7f215414a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53822/0 (socket says 172.21.15.46:53822) 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 -- 172.21.15.46:0/1338240784 learned_addr learned my addr 172.21.15.46:0/1338240784 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21598fc640 1 --2- 172.21.15.46:0/1338240784 >> v2:172.21.15.161:3300/0 conn(0x7f215413ff50 0x7f21541403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21590fb640 1 --2- 172.21.15.46:0/1338240784 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21540bc800 0x7f21540b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 -- 172.21.15.46:0/1338240784 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21540bc800 msgr2=0x7f21540b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 --2- 172.21.15.46:0/1338240784 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21540bc800 0x7f21540b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 -- 172.21.15.46:0/1338240784 >> v2:172.21.15.161:3300/0 conn(0x7f215413ff50 msgr2=0x7f21541403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 --2- 172.21.15.46:0/1338240784 >> v2:172.21.15.161:3300/0 conn(0x7f215413ff50 0x7f21541403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 -- 172.21.15.46:0/1338240784 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540b3bd0 con 0x7f2154149c80 2023-12-16T21:43:23.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21590fb640 1 --2- 172.21.15.46:0/1338240784 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21540bc800 0x7f21540b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:23.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f21588fa640 1 --2- 172.21.15.46:0/1338240784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2154149c80 0x7f215414a0e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f215414aca0 tx=0x7f214401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6037f14ff0a91a6b server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f214bfff640 1 -- 172.21.15.46:0/1338240784 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440ad660 con 0x7f2154149c80 2023-12-16T21:43:23.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.875+0000 7f214bfff640 1 -- 172.21.15.46:0/1338240784 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21440ad7c0 con 0x7f2154149c80 2023-12-16T21:43:23.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f214bfff640 1 -- 172.21.15.46:0/1338240784 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440adb50 con 0x7f2154149c80 2023-12-16T21:43:23.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- 172.21.15.46:0/1338240784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2154149c80 msgr2=0x7f215414a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 --2- 172.21.15.46:0/1338240784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2154149c80 0x7f215414a0e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f215414aca0 tx=0x7f214401c4d0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- 172.21.15.46:0/1338240784 shutdown_connections 2023-12-16T21:43:23.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 --2- 172.21.15.46:0/1338240784 >> v2:172.21.15.161:3300/0 conn(0x7f215413ff50 0x7f21541403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 --2- 172.21.15.46:0/1338240784 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2154149c80 0x7f215414a0e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 --2- 172.21.15.46:0/1338240784 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f21540bc800 0x7f21540b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- 172.21.15.46:0/1338240784 >> 172.21.15.46:0/1338240784 conn(0x7f21540722b0 msgr2=0x7f21540bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:23.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- 172.21.15.46:0/1338240784 shutdown_connections 2023-12-16T21:43:23.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- 172.21.15.46:0/1338240784 wait complete. 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 Processor -- start 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- start start 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21540bc800 0x7f215413d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215413ff50 0x7f2154130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2154149c80 0x7f2154135f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215413f220 con 0x7f215413ff50 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21541df3c0 con 0x7f21540bc800 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21541de8e0 con 0x7f2154149c80 2023-12-16T21:43:23.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215413ff50 0x7f2154130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215413ff50 0x7f2154130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53838/0 (socket says 172.21.15.46:53838) 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 -- 172.21.15.46:0/505135269 learned_addr learned my addr 172.21.15.46:0/505135269 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21590fb640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21540bc800 0x7f215413d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21598fc640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2154149c80 0x7f2154135f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 -- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2154149c80 msgr2=0x7f2154135f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2154149c80 0x7f2154135f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 -- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21540bc800 msgr2=0x7f215413d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21540bc800 0x7f215413d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 -- 172.21.15.46:0/505135269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440094d0 con 0x7f215413ff50 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f21588fa640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215413ff50 0x7f2154130ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f21440b0040 tx=0x7f2144019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f2149ffb640 1 -- 172.21.15.46:0/505135269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440b6640 con 0x7f215413ff50 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f2149ffb640 1 -- 172.21.15.46:0/505135269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21440b67a0 con 0x7f215413ff50 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f2149ffb640 1 -- 172.21.15.46:0/505135269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21440b6b50 con 0x7f215413ff50 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21541debc0 con 0x7f215413ff50 2023-12-16T21:43:23.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.879+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154135320 con 0x7f215413ff50 2023-12-16T21:43:23.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.883+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2154067de0 con 0x7f215413ff50 2023-12-16T21:43:23.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.883+0000 7f2149ffb640 1 -- 172.21.15.46:0/505135269 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f21440b4070 con 0x7f215413ff50 2023-12-16T21:43:23.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.883+0000 7f2149ffb640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2138066470 0x7f2138068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:23.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.883+0000 7f2149ffb640 1 -- 172.21.15.46:0/505135269 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f214411e0c0 con 0x7f215413ff50 2023-12-16T21:43:23.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.883+0000 7f21590fb640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2138066470 0x7f2138068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:23.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.883+0000 7f21590fb640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2138066470 0x7f2138068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2154135870 tx=0x7f2150008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:23.889 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:23.887+0000 7f2149ffb640 1 -- 172.21.15.46:0/505135269 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f21440eb5d0 con 0x7f215413ff50 2023-12-16T21:43:24.038 INFO:tasks.ceph.mds.a.smithi046.stderr:2023-12-16T21:43:24.035+0000 7fd37d85a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-16T21:43:24.044 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.039+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21540b7020 con 0x7f2138066470 2023-12-16T21:43:24.061 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.059+0000 7f2149ffb640 1 -- 172.21.15.46:0/505135269 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249866 (secure 0 0 0) 0x7f215000a670 con 0x7f2138066470 2023-12-16T21:43:24.063 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:24.065 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2138066470 msgr2=0x7f2138068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2138066470 0x7f2138068930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2154135870 tx=0x7f2150008040 comp rx=0 tx=0).stop 2023-12-16T21:43:24.066 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215413ff50 msgr2=0x7f2154130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215413ff50 0x7f2154130ba0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f21440b0040 tx=0x7f2144019270 comp rx=0 tx=0).stop 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 shutdown_connections 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2138066470 0x7f2138068930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2154149c80 0x7f2154135f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f215413ff50 0x7f2154130ba0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 --2- 172.21.15.46:0/505135269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f21540bc800 0x7f215413d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 >> 172.21.15.46:0/505135269 conn(0x7f21540722b0 msgr2=0x7f2154146bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 shutdown_connections 2023-12-16T21:43:24.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.063+0000 7f215b35f640 1 -- 172.21.15.46:0/505135269 wait complete. 2023-12-16T21:43:24.083 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2023-12-16T21:43:22.364480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236232,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738488,"statfs":{"total":773094113280,"available":772852211712,"internally_reserved":0,"allocated":3481600,"data_stored":2008736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":77,"apply_latency_ms":77,"commit_latency_ns":77000000,"apply_latency_ns":77000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.121185"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:19.239844+0000","last_change":"2023-12-16T21:43:19.239844+0000","last_active":"2023-12-16T21:43:19.239844+0000","last_peered":"2023-12-16T21:43:19.239844+0000","last_clean":"2023-12-16T21:43:19.239844+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:19.239844+0000","last_undegraded":"2023-12-16T21:43:19.239844+0000","last_fullsized":"2023-12-16T21:43:19.239844+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-16T21:43:18.236160+0000","last_change":"2023-12-16T21:43:18.236160+0000","last_active":"2023-12-16T21:43:18.236160+0000","last_peered":"2023-12-16T21:43:18.236160+0000","last_clean":"2023-12-16T21:43:18.236160+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-16T21:43:18.236160+0000","last_undegraded":"2023-12-16T21:43:18.236160+0000","last_fullsized":"2023-12-16T21:43:18.236160+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-12-16T21:43:14.028059+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:14.028059+0000","last_peered":"2023-12-16T21:43:14.028059+0000","last_clean":"2023-12-16T21:43:14.028059+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:14.028059+0000","last_undegraded":"2023-12-16T21:43:14.028059+0000","last_fullsized":"2023-12-16T21:43:14.028059+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":880640,"data_stored":693868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.514}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.462}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95099999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.024}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2450000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31464,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340376,"statfs":{"total":96636764160,"available":96604545024,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.135}]}]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31460,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340380,"statfs":{"total":96636764160,"available":96604549120,"internally_reserved":0,"allocated":286720,"data_stored":103500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":880640,"data_stored":693868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.871}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-12-16T21:43:24.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-12-16T21:43:24.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-12-16T21:43:24.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-12-16T21:43:24.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-12-16T21:43:24.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-12-16T21:43:24.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-12-16T21:43:24.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:19.239844+0000', 'last_change': '2023-12-16T21:43:19.239844+0000', 'last_active': '2023-12-16T21:43:19.239844+0000', 'last_peered': '2023-12-16T21:43:19.239844+0000', 'last_clean': '2023-12-16T21:43:19.239844+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:19.239844+0000', 'last_undegraded': '2023-12-16T21:43:19.239844+0000', 'last_fullsized': '2023-12-16T21:43:19.239844+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:19.239844+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-12-16T21:43:24.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-16T21:43:18.236160+0000', 'last_change': '2023-12-16T21:43:18.236160+0000', 'last_active': '2023-12-16T21:43:18.236160+0000', 'last_peered': '2023-12-16T21:43:18.236160+0000', 'last_clean': '2023-12-16T21:43:18.236160+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-16T21:43:18.236160+0000', 'last_undegraded': '2023-12-16T21:43:18.236160+0000', 'last_fullsized': '2023-12-16T21:43:18.236160+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'last_clean_scrub_stamp': '2023-12-16T21:43:18.236160+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-16T21:43:24.104 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-16T21:43:24.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.239+0000 7f0a222ba640 1 Processor -- start 2023-12-16T21:43:24.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.239+0000 7f0a222ba640 1 -- start start 2023-12-16T21:43:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.239+0000 7f0a222ba640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0a1c0bc830 0x7f0a1c0b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.239+0000 7f0a222ba640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a1c149ef0 0x7f0a1c14a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.239+0000 7f0a222ba640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1c140e10 con 0x7f0a1c1402d0 2023-12-16T21:43:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.239+0000 7f0a222ba640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c0b4690 con 0x7f0a1c149ef0 2023-12-16T21:43:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.239+0000 7f0a222ba640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a1c14a8c0 con 0x7f0a1c0bc830 2023-12-16T21:43:24.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a1c149ef0 0x7f0a1c14a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0a1c149ef0 0x7f0a1c14a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53844/0 (socket says 172.21.15.46:53844) 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 -- 172.21.15.46:0/82265650 learned_addr learned my addr 172.21.15.46:0/82265650 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1bfff640 1 --2- 172.21.15.46:0/82265650 >> v2:172.21.15.161:3300/0 conn(0x7f0a1c0bc830 0x7f0a1c0b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 -- 172.21.15.46:0/82265650 >> v2:172.21.15.161:3300/0 conn(0x7f0a1c0bc830 msgr2=0x7f0a1c0b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 --2- 172.21.15.46:0/82265650 >> v2:172.21.15.161:3300/0 conn(0x7f0a1c0bc830 0x7f0a1c0b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 -- 172.21.15.46:0/82265650 >> v1:172.21.15.161:6789/0 conn(0x7f0a1c1402d0 legacy=0x7f0a1c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 -- 172.21.15.46:0/82265650 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a1c0b8740 con 0x7f0a1c149ef0 2023-12-16T21:43:24.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1bfff640 1 --2- 172.21.15.46:0/82265650 >> v2:172.21.15.161:3300/0 conn(0x7f0a1c0bc830 0x7f0a1c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:24.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1b7fe640 1 --2- 172.21.15.46:0/82265650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c14a380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0b8260 tx=0x7f0a0401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f683c471afa0edf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1affd640 1 -- 172.21.15.46:0/82265650 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040a7070 con 0x7f0a1c149ef0 2023-12-16T21:43:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1affd640 1 -- 172.21.15.46:0/82265650 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a040ab430 con 0x7f0a1c149ef0 2023-12-16T21:43:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a1affd640 1 -- 172.21.15.46:0/82265650 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040ab7c0 con 0x7f0a1c149ef0 2023-12-16T21:43:24.248 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 -- 172.21.15.46:0/82265650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 msgr2=0x7f0a1c14a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 --2- 172.21.15.46:0/82265650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c14a380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0b8260 tx=0x7f0a0401c430 comp rx=0 tx=0).stop 2023-12-16T21:43:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 -- 172.21.15.46:0/82265650 shutdown_connections 2023-12-16T21:43:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 --2- 172.21.15.46:0/82265650 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c14a380 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 --2- 172.21.15.46:0/82265650 >> v2:172.21.15.161:3300/0 conn(0x7f0a1c0bc830 0x7f0a1c0b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 -- 172.21.15.46:0/82265650 >> 172.21.15.46:0/82265650 conn(0x7f0a1c072340 msgr2=0x7f0a1c0bbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 -- 172.21.15.46:0/82265650 shutdown_connections 2023-12-16T21:43:24.249 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.243+0000 7f0a222ba640 1 -- 172.21.15.46:0/82265650 wait complete. 2023-12-16T21:43:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 Processor -- start 2023-12-16T21:43:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 -- start start 2023-12-16T21:43:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a1c0bc830 0x7f0a1c1488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a1c1402d0 0x7f0a1c1dede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c1df320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c0b64d0 con 0x7f0a1c149ef0 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c139c50 con 0x7f0a1c1402d0 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c149190 con 0x7f0a1c0bc830 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c1df320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c1df320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53856/0 (socket says 172.21.15.46:53856) 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 -- 172.21.15.46:0/420492825 learned_addr learned my addr 172.21.15.46:0/420492825 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a1b7fe640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a1c1402d0 0x7f0a1c1dede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a1bfff640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a1c0bc830 0x7f0a1c1488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 -- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a1c0bc830 msgr2=0x7f0a1c1488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a1c0bc830 0x7f0a1c1488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 -- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a1c1402d0 msgr2=0x7f0a1c1dede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a1c1402d0 0x7f0a1c1dede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 -- 172.21.15.46:0/420492825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040094d0 con 0x7f0a1c149ef0 2023-12-16T21:43:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a20857640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c1df320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c3c0 tx=0x7f0a1400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a18ff9640 1 -- 172.21.15.46:0/420492825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011790 con 0x7f0a1c149ef0 2023-12-16T21:43:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a18ff9640 1 -- 172.21.15.46:0/420492825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a140118f0 con 0x7f0a1c149ef0 2023-12-16T21:43:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c139970 con 0x7f0a1c149ef0 2023-12-16T21:43:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a18ff9640 1 -- 172.21.15.46:0/420492825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14011ca0 con 0x7f0a1c149ef0 2023-12-16T21:43:24.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c149470 con 0x7f0a1c149ef0 2023-12-16T21:43:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.247+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c067e60 con 0x7f0a1c149ef0 2023-12-16T21:43:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.251+0000 7f0a18ff9640 1 -- 172.21.15.46:0/420492825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f0a1401c460 con 0x7f0a1c149ef0 2023-12-16T21:43:24.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.251+0000 7f0a18ff9640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f09fc0664a0 0x7f09fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.251+0000 7f0a18ff9640 1 -- 172.21.15.46:0/420492825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a14084740 con 0x7f0a1c149ef0 2023-12-16T21:43:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.251+0000 7f0a1bfff640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f09fc0664a0 0x7f09fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.251+0000 7f0a1bfff640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f09fc0664a0 0x7f09fc068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009f70 tx=0x7f0a0c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:24.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.255+0000 7f0a18ff9640 1 -- 172.21.15.46:0/420492825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f0a14051d30 con 0x7f0a1c149ef0 2023-12-16T21:43:24.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.463+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0a1c0679e0 con 0x7f0a1c149ef0 2023-12-16T21:43:24.469 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a18ff9640 1 -- 172.21.15.46:0/420492825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2366 (secure 0 0 0) 0x7f0a14003810 con 0x7f0a1c149ef0 2023-12-16T21:43:24.469 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:24.469 INFO:teuthology.orchestra.run.smithi046.stdout:{"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":68,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":30,"num_osds":8,"num_up_osds":8,"osd_up_since":1702762947,"num_in_osds":8,"osd_in_since":1702762938,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592778,"bytes_used":242368512,"bytes_avail":772851744768,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4863}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-12-16T21:43:22.364572+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f09fc0664a0 msgr2=0x7f09fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f09fc0664a0 0x7f09fc068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009f70 tx=0x7f0a0c00b040 comp rx=0 tx=0).stop 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 msgr2=0x7f0a1c1df320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c1df320 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c3c0 tx=0x7f0a1400c880 comp rx=0 tx=0).stop 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 shutdown_connections 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f09fc0664a0 0x7f09fc068960 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0a1c149ef0 0x7f0a1c1df320 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0a1c1402d0 0x7f0a1c1dede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.472 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 --2- 172.21.15.46:0/420492825 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0a1c0bc830 0x7f0a1c1488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 >> 172.21.15.46:0/420492825 conn(0x7f0a1c072340 msgr2=0x7f0a1c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 shutdown_connections 2023-12-16T21:43:24.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.467+0000 7f0a222ba640 1 -- 172.21.15.46:0/420492825 wait complete. 2023-12-16T21:43:24.489 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-16T21:43:24.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:24.607 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.603+0000 7f2a50261640 1 Processor -- start 2023-12-16T21:43:24.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.603+0000 7f2a50261640 1 -- start start 2023-12-16T21:43:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.603+0000 7f2a50261640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a480b0340 0x7f2a4813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.603+0000 7f2a50261640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a480af2e0 con 0x7f2a4806c920 2023-12-16T21:43:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.603+0000 7f2a50261640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a480b1f50 con 0x7f2a480b3fd0 2023-12-16T21:43:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.603+0000 7f2a50261640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4806cd00 con 0x7f2a480b0340 2023-12-16T21:43:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4dffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2a4806c920 0x7f2a480afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42464/0 (socket says 172.21.15.46:42464) 2023-12-16T21:43:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4dffd640 1 -- 172.21.15.46:0/4235095636 learned_addr learned my addr 172.21.15.46:0/4235095636 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:24.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 613593517 0 0) 0x7f2a480af2e0 con 0x7f2a4806c920 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a34003530 con 0x7f2a4806c920 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4e7fe640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a480b0340 0x7f2a4813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 197763160 0 0) 0x7f2a34003530 con 0x7f2a4806c920 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a480b0340 msgr2=0x7f2a4813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a480b0340 0x7f2a4813ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 >> v1:172.21.15.161:6789/0 conn(0x7f2a480b3fd0 legacy=0x7f2a4813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480af530 con 0x7f2a4806c920 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f2a44001680 con 0x7f2a4806c920 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 >> v1:172.21.15.46:6789/0 conn(0x7f2a4806c920 legacy=0x7f2a480afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a340039b0 0x7f2a34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a34004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a34005190 0x7f2a3400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4806cd00 con 0x7f2a340047d0 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3400bed0 con 0x7f2a340039b0 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3400c060 con 0x7f2a34005190 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a34004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4e7fe640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a34005190 0x7f2a3400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4dffd640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a340039b0 0x7f2a34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4d7fc640 1 -- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a34005190 msgr2=0x7f2a3400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a34005190 0x7f2a3400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4d7fc640 1 -- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a340039b0 msgr2=0x7f2a34003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a340039b0 0x7f2a34003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4d7fc640 1 -- 172.21.15.46:0/4235095636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a34003530 con 0x7f2a340047d0 2023-12-16T21:43:24.613 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a34004c50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001780 tx=0x7f2a3801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c4e1f5128d0f46 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a380052d0 con 0x7f2a340047d0 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a380a7050 con 0x7f2a340047d0 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.607+0000 7f2a4cffb640 1 -- 172.21.15.46:0/4235095636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38003600 con 0x7f2a340047d0 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/4235095636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 msgr2=0x7f2a34004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 --2- 172.21.15.46:0/4235095636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a34004c50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001780 tx=0x7f2a3801c400 comp rx=0 tx=0).stop 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4e7fe640 1 -- 172.21.15.46:0/4235095636 reap_dead start 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/4235095636 shutdown_connections 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/4235095636 >> 172.21.15.46:0/4235095636 conn(0x7f2a480b48e0 msgr2=0x7f2a480b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/4235095636 shutdown_connections 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/4235095636 wait complete. 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 Processor -- start 2023-12-16T21:43:24.614 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- start start 2023-12-16T21:43:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a48140950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a48144ea0 0x7f2a481e0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a48134ed0 0x7f2a48135360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48145320 con 0x7f2a340047d0 2023-12-16T21:43:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a48141830 con 0x7f2a48134ed0 2023-12-16T21:43:24.616 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a481e09b0 con 0x7f2a48144ea0 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a48140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a48140950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53886/0 (socket says 172.21.15.46:53886) 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 -- 172.21.15.46:0/2808080088 learned_addr learned my addr 172.21.15.46:0/2808080088 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4e7fe640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a48134ed0 0x7f2a48135360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a48144ea0 0x7f2a481e0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 -- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a48144ea0 msgr2=0x7f2a481e0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a48144ea0 0x7f2a481e0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 -- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a48134ed0 msgr2=0x7f2a48135360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a48134ed0 0x7f2a48135360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 -- 172.21.15.46:0/2808080088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a48141d40 con 0x7f2a340047d0 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a48144ea0 0x7f2a481e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a4dffd640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a48140950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400d780 tx=0x7f2a4400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a3e7fc640 1 -- 172.21.15.46:0/2808080088 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a440074e0 con 0x7f2a340047d0 2023-12-16T21:43:24.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a3e7fc640 1 -- 172.21.15.46:0/2808080088 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a44007640 con 0x7f2a340047d0 2023-12-16T21:43:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4813fe70 con 0x7f2a340047d0 2023-12-16T21:43:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a3e7fc640 1 -- 172.21.15.46:0/2808080088 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a44004a30 con 0x7f2a340047d0 2023-12-16T21:43:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a48140430 con 0x7f2a340047d0 2023-12-16T21:43:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.611+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a480679e0 con 0x7f2a340047d0 2023-12-16T21:43:24.618 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.615+0000 7f2a3e7fc640 1 -- 172.21.15.46:0/2808080088 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2a44004b90 con 0x7f2a340047d0 2023-12-16T21:43:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.615+0000 7f2a3e7fc640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2a2c066470 0x7f2a2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.615+0000 7f2a3e7fc640 1 -- 172.21.15.46:0/2808080088 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a44083b00 con 0x7f2a340047d0 2023-12-16T21:43:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.615+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2a2c066470 0x7f2a2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:24.619 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.615+0000 7f2a4d7fc640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2a2c066470 0x7f2a2c068930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001750 tx=0x7f2a380011a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:24.622 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.619+0000 7f2a3e7fc640 1 -- 172.21.15.46:0/2808080088 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2a44085e60 con 0x7f2a340047d0 2023-12-16T21:43:24.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.767+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2a48140650 con 0x7f2a2c066470 2023-12-16T21:43:24.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.787+0000 7f2a3e7fc640 1 -- 172.21.15.46:0/2808080088 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264659 (secure 0 0 0) 0x7f2a48140650 con 0x7f2a2c066470 2023-12-16T21:43:24.792 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:24.794 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2a2c066470 msgr2=0x7f2a2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2a2c066470 0x7f2a2c068930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2a38001750 tx=0x7f2a380011a0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 msgr2=0x7f2a48140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:24.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a48140950 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2a4400d780 tx=0x7f2a4400dc40 comp rx=0 tx=0).stop 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 shutdown_connections 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2a2c066470 0x7f2a2c068930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2a48134ed0 0x7f2a48135360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2a48144ea0 0x7f2a481e0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 --2- 172.21.15.46:0/2808080088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2a340047d0 0x7f2a48140950 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 >> 172.21.15.46:0/2808080088 conn(0x7f2a480b48e0 msgr2=0x7f2a480b6480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 shutdown_connections 2023-12-16T21:43:24.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:24.791+0000 7f2a50261640 1 -- 172.21.15.46:0/2808080088 wait complete. 2023-12-16T21:43:24.810 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-12-16T21:43:24.365113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236688,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738032,"statfs":{"total":773094113280,"available":772851744768,"internally_reserved":0,"allocated":3940352,"data_stored":2214828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001766"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370766+0000","last_change":"2023-12-16T21:43:20.325568+0000","last_active":"2023-12-16T21:43:21.370766+0000","last_peered":"2023-12-16T21:43:21.370766+0000","last_clean":"2023-12-16T21:43:21.370766+0000","last_became_active":"2023-12-16T21:43:20.324943+0000","last_became_peered":"2023-12-16T21:43:20.324943+0000","last_unstale":"2023-12-16T21:43:21.370766+0000","last_undegraded":"2023-12-16T21:43:21.370766+0000","last_fullsized":"2023-12-16T21:43:21.370766+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:39.005796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369403+0000","last_change":"2023-12-16T21:43:20.305510+0000","last_active":"2023-12-16T21:43:21.369403+0000","last_peered":"2023-12-16T21:43:21.369403+0000","last_clean":"2023-12-16T21:43:21.369403+0000","last_became_active":"2023-12-16T21:43:20.304590+0000","last_became_peered":"2023-12-16T21:43:20.304590+0000","last_unstale":"2023-12-16T21:43:21.369403+0000","last_undegraded":"2023-12-16T21:43:21.369403+0000","last_fullsized":"2023-12-16T21:43:21.369403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:32:36.029124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370894+0000","last_change":"2023-12-16T21:43:20.322443+0000","last_active":"2023-12-16T21:43:21.370894+0000","last_peered":"2023-12-16T21:43:21.370894+0000","last_clean":"2023-12-16T21:43:21.370894+0000","last_became_active":"2023-12-16T21:43:20.321782+0000","last_became_peered":"2023-12-16T21:43:20.321782+0000","last_unstale":"2023-12-16T21:43:21.370894+0000","last_undegraded":"2023-12-16T21:43:21.370894+0000","last_fullsized":"2023-12-16T21:43:21.370894+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368640+0000","last_change":"2023-12-16T21:43:20.318503+0000","last_active":"2023-12-16T21:43:21.368640+0000","last_peered":"2023-12-16T21:43:21.368640+0000","last_clean":"2023-12-16T21:43:21.368640+0000","last_became_active":"2023-12-16T21:43:20.317848+0000","last_became_peered":"2023-12-16T21:43:20.317848+0000","last_unstale":"2023-12-16T21:43:21.368640+0000","last_undegraded":"2023-12-16T21:43:21.368640+0000","last_fullsized":"2023-12-16T21:43:21.368640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:32:37.254923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367453+0000","last_change":"2023-12-16T21:43:20.313033+0000","last_active":"2023-12-16T21:43:21.367453+0000","last_peered":"2023-12-16T21:43:21.367453+0000","last_clean":"2023-12-16T21:43:21.367453+0000","last_became_active":"2023-12-16T21:43:20.311438+0000","last_became_peered":"2023-12-16T21:43:20.311438+0000","last_unstale":"2023-12-16T21:43:21.367453+0000","last_undegraded":"2023-12-16T21:43:21.367453+0000","last_fullsized":"2023-12-16T21:43:21.367453+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:49:33.889831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368240+0000","last_change":"2023-12-16T21:43:20.325278+0000","last_active":"2023-12-16T21:43:21.368240+0000","last_peered":"2023-12-16T21:43:21.368240+0000","last_clean":"2023-12-16T21:43:21.368240+0000","last_became_active":"2023-12-16T21:43:20.324935+0000","last_became_peered":"2023-12-16T21:43:20.324935+0000","last_unstale":"2023-12-16T21:43:21.368240+0000","last_undegraded":"2023-12-16T21:43:21.368240+0000","last_fullsized":"2023-12-16T21:43:21.368240+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:15:29.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278397+0000","last_change":"2023-12-16T21:43:20.300989+0000","last_active":"2023-12-16T21:43:21.278397+0000","last_peered":"2023-12-16T21:43:21.278397+0000","last_clean":"2023-12-16T21:43:21.278397+0000","last_became_active":"2023-12-16T21:43:20.300268+0000","last_became_peered":"2023-12-16T21:43:20.300268+0000","last_unstale":"2023-12-16T21:43:21.278397+0000","last_undegraded":"2023-12-16T21:43:21.278397+0000","last_fullsized":"2023-12-16T21:43:21.278397+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:46:42.032255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277462+0000","last_change":"2023-12-16T21:43:20.299012+0000","last_active":"2023-12-16T21:43:21.277462+0000","last_peered":"2023-12-16T21:43:21.277462+0000","last_clean":"2023-12-16T21:43:21.277462+0000","last_became_active":"2023-12-16T21:43:20.298084+0000","last_became_peered":"2023-12-16T21:43:20.298084+0000","last_unstale":"2023-12-16T21:43:21.277462+0000","last_undegraded":"2023-12-16T21:43:21.277462+0000","last_fullsized":"2023-12-16T21:43:21.277462+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280405+0000","last_change":"2023-12-16T21:43:20.318994+0000","last_active":"2023-12-16T21:43:21.280405+0000","last_peered":"2023-12-16T21:43:21.280405+0000","last_clean":"2023-12-16T21:43:21.280405+0000","last_became_active":"2023-12-16T21:43:20.318279+0000","last_became_peered":"2023-12-16T21:43:20.318279+0000","last_unstale":"2023-12-16T21:43:21.280405+0000","last_undegraded":"2023-12-16T21:43:21.280405+0000","last_fullsized":"2023-12-16T21:43:21.280405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:32:37.254923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369410+0000","last_change":"2023-12-16T21:43:20.317059+0000","last_active":"2023-12-16T21:43:21.369410+0000","last_peered":"2023-12-16T21:43:21.369410+0000","last_clean":"2023-12-16T21:43:21.369410+0000","last_became_active":"2023-12-16T21:43:20.316318+0000","last_became_peered":"2023-12-16T21:43:20.316318+0000","last_unstale":"2023-12-16T21:43:21.369410+0000","last_undegraded":"2023-12-16T21:43:21.369410+0000","last_fullsized":"2023-12-16T21:43:21.369410+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:16.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278127+0000","last_change":"2023-12-16T21:43:20.328830+0000","last_active":"2023-12-16T21:43:21.278127+0000","last_peered":"2023-12-16T21:43:21.278127+0000","last_clean":"2023-12-16T21:43:21.278127+0000","last_became_active":"2023-12-16T21:43:20.328205+0000","last_became_peered":"2023-12-16T21:43:20.328205+0000","last_unstale":"2023-12-16T21:43:21.278127+0000","last_undegraded":"2023-12-16T21:43:21.278127+0000","last_fullsized":"2023-12-16T21:43:21.278127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:32:36.029124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371475+0000","last_change":"2023-12-16T21:43:20.323210+0000","last_active":"2023-12-16T21:43:21.371475+0000","last_peered":"2023-12-16T21:43:21.371475+0000","last_clean":"2023-12-16T21:43:21.371475+0000","last_became_active":"2023-12-16T21:43:20.320416+0000","last_became_peered":"2023-12-16T21:43:20.320416+0000","last_unstale":"2023-12-16T21:43:21.371475+0000","last_undegraded":"2023-12-16T21:43:21.371475+0000","last_fullsized":"2023-12-16T21:43:21.371475+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:21:43.837309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367640+0000","last_change":"2023-12-16T21:43:20.317694+0000","last_active":"2023-12-16T21:43:21.367640+0000","last_peered":"2023-12-16T21:43:21.367640+0000","last_clean":"2023-12-16T21:43:21.367640+0000","last_became_active":"2023-12-16T21:43:20.316996+0000","last_became_peered":"2023-12-16T21:43:20.316996+0000","last_unstale":"2023-12-16T21:43:21.367640+0000","last_undegraded":"2023-12-16T21:43:21.367640+0000","last_fullsized":"2023-12-16T21:43:21.367640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:00:23.034732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368436+0000","last_change":"2023-12-16T21:43:20.304950+0000","last_active":"2023-12-16T21:43:21.368436+0000","last_peered":"2023-12-16T21:43:21.368436+0000","last_clean":"2023-12-16T21:43:21.368436+0000","last_became_active":"2023-12-16T21:43:20.304097+0000","last_became_peered":"2023-12-16T21:43:20.304097+0000","last_unstale":"2023-12-16T21:43:21.368436+0000","last_undegraded":"2023-12-16T21:43:21.368436+0000","last_fullsized":"2023-12-16T21:43:21.368436+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:15:32.234255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370718+0000","last_change":"2023-12-16T21:43:20.319973+0000","last_active":"2023-12-16T21:43:21.370718+0000","last_peered":"2023-12-16T21:43:21.370718+0000","last_clean":"2023-12-16T21:43:21.370718+0000","last_became_active":"2023-12-16T21:43:20.319308+0000","last_became_peered":"2023-12-16T21:43:20.319308+0000","last_unstale":"2023-12-16T21:43:21.370718+0000","last_undegraded":"2023-12-16T21:43:21.370718+0000","last_fullsized":"2023-12-16T21:43:21.370718+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:16.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278182+0000","last_change":"2023-12-16T21:43:20.318376+0000","last_active":"2023-12-16T21:43:21.278182+0000","last_peered":"2023-12-16T21:43:21.278182+0000","last_clean":"2023-12-16T21:43:21.278182+0000","last_became_active":"2023-12-16T21:43:20.317573+0000","last_became_peered":"2023-12-16T21:43:20.317573+0000","last_unstale":"2023-12-16T21:43:21.278182+0000","last_undegraded":"2023-12-16T21:43:21.278182+0000","last_fullsized":"2023-12-16T21:43:21.278182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:39:10.806527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278890+0000","last_change":"2023-12-16T21:43:20.311318+0000","last_active":"2023-12-16T21:43:21.278890+0000","last_peered":"2023-12-16T21:43:21.278890+0000","last_clean":"2023-12-16T21:43:21.278890+0000","last_became_active":"2023-12-16T21:43:20.310591+0000","last_became_peered":"2023-12-16T21:43:20.310591+0000","last_unstale":"2023-12-16T21:43:21.278890+0000","last_undegraded":"2023-12-16T21:43:21.278890+0000","last_fullsized":"2023-12-16T21:43:21.278890+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:52:20.633557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369493+0000","last_change":"2023-12-16T21:43:20.321144+0000","last_active":"2023-12-16T21:43:21.369493+0000","last_peered":"2023-12-16T21:43:21.369493+0000","last_clean":"2023-12-16T21:43:21.369493+0000","last_became_active":"2023-12-16T21:43:20.320838+0000","last_became_peered":"2023-12-16T21:43:20.320838+0000","last_unstale":"2023-12-16T21:43:21.369493+0000","last_undegraded":"2023-12-16T21:43:21.369493+0000","last_fullsized":"2023-12-16T21:43:21.369493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:39:10.806527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369713+0000","last_change":"2023-12-16T21:43:20.327347+0000","last_active":"2023-12-16T21:43:21.369713+0000","last_peered":"2023-12-16T21:43:21.369713+0000","last_clean":"2023-12-16T21:43:21.369713+0000","last_became_active":"2023-12-16T21:43:20.326953+0000","last_became_peered":"2023-12-16T21:43:20.326953+0000","last_unstale":"2023-12-16T21:43:21.369713+0000","last_undegraded":"2023-12-16T21:43:21.369713+0000","last_fullsized":"2023-12-16T21:43:21.369713+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:52:20.633557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367790+0000","last_change":"2023-12-16T21:43:20.324821+0000","last_active":"2023-12-16T21:43:21.367790+0000","last_peered":"2023-12-16T21:43:21.367790+0000","last_clean":"2023-12-16T21:43:21.367790+0000","last_became_active":"2023-12-16T21:43:20.324069+0000","last_became_peered":"2023-12-16T21:43:20.324069+0000","last_unstale":"2023-12-16T21:43:21.367790+0000","last_undegraded":"2023-12-16T21:43:21.367790+0000","last_fullsized":"2023-12-16T21:43:21.367790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:00:46.209082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279491+0000","last_change":"2023-12-16T21:43:20.302529+0000","last_active":"2023-12-16T21:43:21.279491+0000","last_peered":"2023-12-16T21:43:21.279491+0000","last_clean":"2023-12-16T21:43:21.279491+0000","last_became_active":"2023-12-16T21:43:20.302091+0000","last_became_peered":"2023-12-16T21:43:20.302091+0000","last_unstale":"2023-12-16T21:43:21.279491+0000","last_undegraded":"2023-12-16T21:43:21.279491+0000","last_fullsized":"2023-12-16T21:43:21.279491+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:43.887172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279873+0000","last_change":"2023-12-16T21:43:20.302223+0000","last_active":"2023-12-16T21:43:21.279873+0000","last_peered":"2023-12-16T21:43:21.279873+0000","last_clean":"2023-12-16T21:43:21.279873+0000","last_became_active":"2023-12-16T21:43:20.301339+0000","last_became_peered":"2023-12-16T21:43:20.301339+0000","last_unstale":"2023-12-16T21:43:21.279873+0000","last_undegraded":"2023-12-16T21:43:21.279873+0000","last_fullsized":"2023-12-16T21:43:21.279873+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:49:43.351056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.274882+0000","last_change":"2023-12-16T21:43:20.314553+0000","last_active":"2023-12-16T21:43:21.274882+0000","last_peered":"2023-12-16T21:43:21.274882+0000","last_clean":"2023-12-16T21:43:21.274882+0000","last_became_active":"2023-12-16T21:43:20.313906+0000","last_became_peered":"2023-12-16T21:43:20.313906+0000","last_unstale":"2023-12-16T21:43:21.274882+0000","last_undegraded":"2023-12-16T21:43:21.274882+0000","last_fullsized":"2023-12-16T21:43:21.274882+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:41.036474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370150+0000","last_change":"2023-12-16T21:43:20.319665+0000","last_active":"2023-12-16T21:43:21.370150+0000","last_peered":"2023-12-16T21:43:21.370150+0000","last_clean":"2023-12-16T21:43:21.370150+0000","last_became_active":"2023-12-16T21:43:20.319110+0000","last_became_peered":"2023-12-16T21:43:20.319110+0000","last_unstale":"2023-12-16T21:43:21.370150+0000","last_undegraded":"2023-12-16T21:43:21.370150+0000","last_fullsized":"2023-12-16T21:43:21.370150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:25:02.272170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369993+0000","last_change":"2023-12-16T21:43:20.317707+0000","last_active":"2023-12-16T21:43:21.369993+0000","last_peered":"2023-12-16T21:43:21.369993+0000","last_clean":"2023-12-16T21:43:21.369993+0000","last_became_active":"2023-12-16T21:43:20.317112+0000","last_became_peered":"2023-12-16T21:43:20.317112+0000","last_unstale":"2023-12-16T21:43:21.369993+0000","last_undegraded":"2023-12-16T21:43:21.369993+0000","last_fullsized":"2023-12-16T21:43:21.369993+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369583+0000","last_change":"2023-12-16T21:43:20.312988+0000","last_active":"2023-12-16T21:43:21.369583+0000","last_peered":"2023-12-16T21:43:21.369583+0000","last_clean":"2023-12-16T21:43:21.369583+0000","last_became_active":"2023-12-16T21:43:20.312213+0000","last_became_peered":"2023-12-16T21:43:20.312213+0000","last_unstale":"2023-12-16T21:43:21.369583+0000","last_undegraded":"2023-12-16T21:43:21.369583+0000","last_fullsized":"2023-12-16T21:43:21.369583+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:46:42.032255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369166+0000","last_change":"2023-12-16T21:43:20.317515+0000","last_active":"2023-12-16T21:43:21.369166+0000","last_peered":"2023-12-16T21:43:21.369166+0000","last_clean":"2023-12-16T21:43:21.369166+0000","last_became_active":"2023-12-16T21:43:20.316614+0000","last_became_peered":"2023-12-16T21:43:20.316614+0000","last_unstale":"2023-12-16T21:43:21.369166+0000","last_undegraded":"2023-12-16T21:43:21.369166+0000","last_fullsized":"2023-12-16T21:43:21.369166+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:25:02.272170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371832+0000","last_change":"2023-12-16T21:43:20.319882+0000","last_active":"2023-12-16T21:43:21.371832+0000","last_peered":"2023-12-16T21:43:21.371832+0000","last_clean":"2023-12-16T21:43:21.371832+0000","last_became_active":"2023-12-16T21:43:20.319160+0000","last_became_peered":"2023-12-16T21:43:20.319160+0000","last_unstale":"2023-12-16T21:43:21.371832+0000","last_undegraded":"2023-12-16T21:43:21.371832+0000","last_fullsized":"2023-12-16T21:43:21.371832+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:32:36.029124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279521+0000","last_change":"2023-12-16T21:43:20.317629+0000","last_active":"2023-12-16T21:43:21.279521+0000","last_peered":"2023-12-16T21:43:21.279521+0000","last_clean":"2023-12-16T21:43:21.279521+0000","last_became_active":"2023-12-16T21:43:20.316933+0000","last_became_peered":"2023-12-16T21:43:20.316933+0000","last_unstale":"2023-12-16T21:43:21.279521+0000","last_undegraded":"2023-12-16T21:43:21.279521+0000","last_fullsized":"2023-12-16T21:43:21.279521+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:38:54.825810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277539+0000","last_change":"2023-12-16T21:43:20.310554+0000","last_active":"2023-12-16T21:43:21.277539+0000","last_peered":"2023-12-16T21:43:21.277539+0000","last_clean":"2023-12-16T21:43:21.277539+0000","last_became_active":"2023-12-16T21:43:20.310080+0000","last_became_peered":"2023-12-16T21:43:20.310080+0000","last_unstale":"2023-12-16T21:43:21.277539+0000","last_undegraded":"2023-12-16T21:43:21.277539+0000","last_fullsized":"2023-12-16T21:43:21.277539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:15:29.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370263+0000","last_change":"2023-12-16T21:43:20.305440+0000","last_active":"2023-12-16T21:43:21.370263+0000","last_peered":"2023-12-16T21:43:21.370263+0000","last_clean":"2023-12-16T21:43:21.370263+0000","last_became_active":"2023-12-16T21:43:20.304740+0000","last_became_peered":"2023-12-16T21:43:20.304740+0000","last_unstale":"2023-12-16T21:43:21.370263+0000","last_undegraded":"2023-12-16T21:43:21.370263+0000","last_fullsized":"2023-12-16T21:43:21.370263+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:21:20.437604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277815+0000","last_change":"2023-12-16T21:43:20.306619+0000","last_active":"2023-12-16T21:43:21.277815+0000","last_peered":"2023-12-16T21:43:21.277815+0000","last_clean":"2023-12-16T21:43:21.277815+0000","last_became_active":"2023-12-16T21:43:20.305996+0000","last_became_peered":"2023-12-16T21:43:20.305996+0000","last_unstale":"2023-12-16T21:43:21.277815+0000","last_undegraded":"2023-12-16T21:43:21.277815+0000","last_fullsized":"2023-12-16T21:43:21.277815+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:15:32.234255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275176+0000","last_change":"2023-12-16T21:43:20.311721+0000","last_active":"2023-12-16T21:43:21.275176+0000","last_peered":"2023-12-16T21:43:21.275176+0000","last_clean":"2023-12-16T21:43:21.275176+0000","last_became_active":"2023-12-16T21:43:20.311029+0000","last_became_peered":"2023-12-16T21:43:20.311029+0000","last_unstale":"2023-12-16T21:43:21.275176+0000","last_undegraded":"2023-12-16T21:43:21.275176+0000","last_fullsized":"2023-12-16T21:43:21.275176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:39.005796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371050+0000","last_change":"2023-12-16T21:43:19.288334+0000","last_active":"2023-12-16T21:43:21.371050+0000","last_peered":"2023-12-16T21:43:21.371050+0000","last_clean":"2023-12-16T21:43:21.371050+0000","last_became_active":"2023-12-16T21:43:19.287636+0000","last_became_peered":"2023-12-16T21:43:19.287636+0000","last_unstale":"2023-12-16T21:43:21.371050+0000","last_undegraded":"2023-12-16T21:43:21.371050+0000","last_fullsized":"2023-12-16T21:43:21.371050+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:23:17.386717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370936+0000","last_change":"2023-12-16T21:43:20.310784+0000","last_active":"2023-12-16T21:43:21.370936+0000","last_peered":"2023-12-16T21:43:21.370936+0000","last_clean":"2023-12-16T21:43:21.370936+0000","last_became_active":"2023-12-16T21:43:20.310062+0000","last_became_peered":"2023-12-16T21:43:20.310062+0000","last_unstale":"2023-12-16T21:43:21.370936+0000","last_undegraded":"2023-12-16T21:43:21.370936+0000","last_fullsized":"2023-12-16T21:43:21.370936+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:43.887172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275455+0000","last_change":"2023-12-16T21:43:19.276377+0000","last_active":"2023-12-16T21:43:21.275455+0000","last_peered":"2023-12-16T21:43:21.275455+0000","last_clean":"2023-12-16T21:43:21.275455+0000","last_became_active":"2023-12-16T21:43:19.275793+0000","last_became_peered":"2023-12-16T21:43:19.275793+0000","last_unstale":"2023-12-16T21:43:21.275455+0000","last_undegraded":"2023-12-16T21:43:21.275455+0000","last_fullsized":"2023-12-16T21:43:21.275455+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:27:20.836500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280256+0000","last_change":"2023-12-16T21:43:20.316280+0000","last_active":"2023-12-16T21:43:21.280256+0000","last_peered":"2023-12-16T21:43:21.280256+0000","last_clean":"2023-12-16T21:43:21.280256+0000","last_became_active":"2023-12-16T21:43:20.315797+0000","last_became_peered":"2023-12-16T21:43:20.315797+0000","last_unstale":"2023-12-16T21:43:21.280256+0000","last_undegraded":"2023-12-16T21:43:21.280256+0000","last_fullsized":"2023-12-16T21:43:21.280256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:00:19.396579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369788+0000","last_change":"2023-12-16T21:43:19.279704+0000","last_active":"2023-12-16T21:43:21.369788+0000","last_peered":"2023-12-16T21:43:21.369788+0000","last_clean":"2023-12-16T21:43:21.369788+0000","last_became_active":"2023-12-16T21:43:19.279378+0000","last_became_peered":"2023-12-16T21:43:19.279378+0000","last_unstale":"2023-12-16T21:43:21.369788+0000","last_undegraded":"2023-12-16T21:43:21.369788+0000","last_fullsized":"2023-12-16T21:43:21.369788+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:40:30.337768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369255+0000","last_change":"2023-12-16T21:43:20.318988+0000","last_active":"2023-12-16T21:43:21.369255+0000","last_peered":"2023-12-16T21:43:21.369255+0000","last_clean":"2023-12-16T21:43:21.369255+0000","last_became_active":"2023-12-16T21:43:20.318381+0000","last_became_peered":"2023-12-16T21:43:20.318381+0000","last_unstale":"2023-12-16T21:43:21.369255+0000","last_undegraded":"2023-12-16T21:43:21.369255+0000","last_fullsized":"2023-12-16T21:43:21.369255+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:40:31.341452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367919+0000","last_change":"2023-12-16T21:43:19.267679+0000","last_active":"2023-12-16T21:43:21.367919+0000","last_peered":"2023-12-16T21:43:21.367919+0000","last_clean":"2023-12-16T21:43:21.367919+0000","last_became_active":"2023-12-16T21:43:19.267292+0000","last_became_peered":"2023-12-16T21:43:19.267292+0000","last_unstale":"2023-12-16T21:43:21.367919+0000","last_undegraded":"2023-12-16T21:43:21.367919+0000","last_fullsized":"2023-12-16T21:43:21.367919+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:51:27.921297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277959+0000","last_change":"2023-12-16T21:43:20.311672+0000","last_active":"2023-12-16T21:43:21.277959+0000","last_peered":"2023-12-16T21:43:21.277959+0000","last_clean":"2023-12-16T21:43:21.277959+0000","last_became_active":"2023-12-16T21:43:20.310932+0000","last_became_peered":"2023-12-16T21:43:20.310932+0000","last_unstale":"2023-12-16T21:43:21.277959+0000","last_undegraded":"2023-12-16T21:43:21.277959+0000","last_fullsized":"2023-12-16T21:43:21.277959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:21:43.837309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279750+0000","last_change":"2023-12-16T21:43:19.288807+0000","last_active":"2023-12-16T21:43:21.279750+0000","last_peered":"2023-12-16T21:43:21.279750+0000","last_clean":"2023-12-16T21:43:21.279750+0000","last_became_active":"2023-12-16T21:43:19.284430+0000","last_became_peered":"2023-12-16T21:43:19.284430+0000","last_unstale":"2023-12-16T21:43:21.279750+0000","last_undegraded":"2023-12-16T21:43:21.279750+0000","last_fullsized":"2023-12-16T21:43:21.279750+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:23:17.386717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371439+0000","last_change":"2023-12-16T21:43:20.319794+0000","last_active":"2023-12-16T21:43:21.371439+0000","last_peered":"2023-12-16T21:43:21.371439+0000","last_clean":"2023-12-16T21:43:21.371439+0000","last_became_active":"2023-12-16T21:43:20.319191+0000","last_became_peered":"2023-12-16T21:43:20.319191+0000","last_unstale":"2023-12-16T21:43:21.371439+0000","last_undegraded":"2023-12-16T21:43:21.371439+0000","last_fullsized":"2023-12-16T21:43:21.371439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:15:32.234255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279764+0000","last_change":"2023-12-16T21:43:19.283093+0000","last_active":"2023-12-16T21:43:21.279764+0000","last_peered":"2023-12-16T21:43:21.279764+0000","last_clean":"2023-12-16T21:43:21.279764+0000","last_became_active":"2023-12-16T21:43:19.282199+0000","last_became_peered":"2023-12-16T21:43:19.282199+0000","last_unstale":"2023-12-16T21:43:21.279764+0000","last_undegraded":"2023-12-16T21:43:21.279764+0000","last_fullsized":"2023-12-16T21:43:21.279764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:21:19.433920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278956+0000","last_change":"2023-12-16T21:43:20.304787+0000","last_active":"2023-12-16T21:43:21.278956+0000","last_peered":"2023-12-16T21:43:21.278956+0000","last_clean":"2023-12-16T21:43:21.278956+0000","last_became_active":"2023-12-16T21:43:20.303151+0000","last_became_peered":"2023-12-16T21:43:20.303151+0000","last_unstale":"2023-12-16T21:43:21.278956+0000","last_undegraded":"2023-12-16T21:43:21.278956+0000","last_fullsized":"2023-12-16T21:43:21.278956+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:41.036474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367807+0000","last_change":"2023-12-16T21:43:19.279774+0000","last_active":"2023-12-16T21:43:21.367807+0000","last_peered":"2023-12-16T21:43:21.367807+0000","last_clean":"2023-12-16T21:43:21.367807+0000","last_became_active":"2023-12-16T21:43:19.279328+0000","last_became_peered":"2023-12-16T21:43:19.279328+0000","last_unstale":"2023-12-16T21:43:21.367807+0000","last_undegraded":"2023-12-16T21:43:21.367807+0000","last_fullsized":"2023-12-16T21:43:21.367807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:00:18.392895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368029+0000","last_change":"2023-12-16T21:43:20.325276+0000","last_active":"2023-12-16T21:43:21.368029+0000","last_peered":"2023-12-16T21:43:21.368029+0000","last_clean":"2023-12-16T21:43:21.368029+0000","last_became_active":"2023-12-16T21:43:20.323914+0000","last_became_peered":"2023-12-16T21:43:20.323914+0000","last_unstale":"2023-12-16T21:43:21.368029+0000","last_undegraded":"2023-12-16T21:43:21.368029+0000","last_fullsized":"2023-12-16T21:43:21.368029+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:49:43.351056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279054+0000","last_change":"2023-12-16T21:43:19.284383+0000","last_active":"2023-12-16T21:43:21.279054+0000","last_peered":"2023-12-16T21:43:21.279054+0000","last_clean":"2023-12-16T21:43:21.279054+0000","last_became_active":"2023-12-16T21:43:19.283848+0000","last_became_peered":"2023-12-16T21:43:19.283848+0000","last_unstale":"2023-12-16T21:43:21.279054+0000","last_undegraded":"2023-12-16T21:43:21.279054+0000","last_fullsized":"2023-12-16T21:43:21.279054+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:48:25.390230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371320+0000","last_change":"2023-12-16T21:43:20.321562+0000","last_active":"2023-12-16T21:43:21.371320+0000","last_peered":"2023-12-16T21:43:21.371320+0000","last_clean":"2023-12-16T21:43:21.371320+0000","last_became_active":"2023-12-16T21:43:20.321119+0000","last_became_peered":"2023-12-16T21:43:20.321119+0000","last_unstale":"2023-12-16T21:43:21.371320+0000","last_undegraded":"2023-12-16T21:43:21.371320+0000","last_fullsized":"2023-12-16T21:43:21.371320+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:15:29.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368984+0000","last_change":"2023-12-16T21:43:19.284609+0000","last_active":"2023-12-16T21:43:21.368984+0000","last_peered":"2023-12-16T21:43:21.368984+0000","last_clean":"2023-12-16T21:43:21.368984+0000","last_became_active":"2023-12-16T21:43:19.284250+0000","last_became_peered":"2023-12-16T21:43:19.284250+0000","last_unstale":"2023-12-16T21:43:21.368984+0000","last_undegraded":"2023-12-16T21:43:21.368984+0000","last_fullsized":"2023-12-16T21:43:21.368984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:15.882102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369041+0000","last_change":"2023-12-16T21:43:20.321125+0000","last_active":"2023-12-16T21:43:21.369041+0000","last_peered":"2023-12-16T21:43:21.369041+0000","last_clean":"2023-12-16T21:43:21.369041+0000","last_became_active":"2023-12-16T21:43:20.320390+0000","last_became_peered":"2023-12-16T21:43:20.320390+0000","last_unstale":"2023-12-16T21:43:21.369041+0000","last_undegraded":"2023-12-16T21:43:21.369041+0000","last_fullsized":"2023-12-16T21:43:21.369041+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:21:43.837309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275431+0000","last_change":"2023-12-16T21:43:19.282877+0000","last_active":"2023-12-16T21:43:21.275431+0000","last_peered":"2023-12-16T21:43:21.275431+0000","last_clean":"2023-12-16T21:43:21.275431+0000","last_became_active":"2023-12-16T21:43:19.282447+0000","last_became_peered":"2023-12-16T21:43:19.282447+0000","last_unstale":"2023-12-16T21:43:21.275431+0000","last_undegraded":"2023-12-16T21:43:21.275431+0000","last_fullsized":"2023-12-16T21:43:21.275431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:23:17.386717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368271+0000","last_change":"2023-12-16T21:43:20.313362+0000","last_active":"2023-12-16T21:43:21.368271+0000","last_peered":"2023-12-16T21:43:21.368271+0000","last_clean":"2023-12-16T21:43:21.368271+0000","last_became_active":"2023-12-16T21:43:20.312656+0000","last_became_peered":"2023-12-16T21:43:20.312656+0000","last_unstale":"2023-12-16T21:43:21.368271+0000","last_undegraded":"2023-12-16T21:43:21.368271+0000","last_fullsized":"2023-12-16T21:43:21.368271+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:32:57.024768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371150+0000","last_change":"2023-12-16T21:43:19.284074+0000","last_active":"2023-12-16T21:43:21.371150+0000","last_peered":"2023-12-16T21:43:21.371150+0000","last_clean":"2023-12-16T21:43:21.371150+0000","last_became_active":"2023-12-16T21:43:19.283391+0000","last_became_peered":"2023-12-16T21:43:19.283391+0000","last_unstale":"2023-12-16T21:43:21.371150+0000","last_undegraded":"2023-12-16T21:43:21.371150+0000","last_fullsized":"2023-12-16T21:43:21.371150+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:39:42.338944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370540+0000","last_change":"2023-12-16T21:43:20.328772+0000","last_active":"2023-12-16T21:43:21.370540+0000","last_peered":"2023-12-16T21:43:21.370540+0000","last_clean":"2023-12-16T21:43:21.370540+0000","last_became_active":"2023-12-16T21:43:20.328229+0000","last_became_peered":"2023-12-16T21:43:20.328229+0000","last_unstale":"2023-12-16T21:43:21.370540+0000","last_undegraded":"2023-12-16T21:43:21.370540+0000","last_fullsized":"2023-12-16T21:43:21.370540+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:40:31.341452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368501+0000","last_change":"2023-12-16T21:43:19.283068+0000","last_active":"2023-12-16T21:43:21.368501+0000","last_peered":"2023-12-16T21:43:21.368501+0000","last_clean":"2023-12-16T21:43:21.368501+0000","last_became_active":"2023-12-16T21:43:19.281758+0000","last_became_peered":"2023-12-16T21:43:19.281758+0000","last_unstale":"2023-12-16T21:43:21.368501+0000","last_undegraded":"2023-12-16T21:43:21.368501+0000","last_fullsized":"2023-12-16T21:43:21.368501+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:23:06.196965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368398+0000","last_change":"2023-12-16T21:43:20.323107+0000","last_active":"2023-12-16T21:43:21.368398+0000","last_peered":"2023-12-16T21:43:21.368398+0000","last_clean":"2023-12-16T21:43:21.368398+0000","last_became_active":"2023-12-16T21:43:20.322469+0000","last_became_peered":"2023-12-16T21:43:20.322469+0000","last_unstale":"2023-12-16T21:43:21.368398+0000","last_undegraded":"2023-12-16T21:43:21.368398+0000","last_fullsized":"2023-12-16T21:43:21.368398+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:00:19.396579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279753+0000","last_change":"2023-12-16T21:43:19.288795+0000","last_active":"2023-12-16T21:43:21.279753+0000","last_peered":"2023-12-16T21:43:21.279753+0000","last_clean":"2023-12-16T21:43:21.279753+0000","last_became_active":"2023-12-16T21:43:19.284406+0000","last_became_peered":"2023-12-16T21:43:19.284406+0000","last_unstale":"2023-12-16T21:43:21.279753+0000","last_undegraded":"2023-12-16T21:43:21.279753+0000","last_fullsized":"2023-12-16T21:43:21.279753+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:57:15.015059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369041+0000","last_change":"2023-12-16T21:43:20.310442+0000","last_active":"2023-12-16T21:43:21.369041+0000","last_peered":"2023-12-16T21:43:21.369041+0000","last_clean":"2023-12-16T21:43:21.369041+0000","last_became_active":"2023-12-16T21:43:20.309658+0000","last_became_peered":"2023-12-16T21:43:20.309658+0000","last_unstale":"2023-12-16T21:43:21.369041+0000","last_undegraded":"2023-12-16T21:43:21.369041+0000","last_fullsized":"2023-12-16T21:43:21.369041+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:48:13.371146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368434+0000","last_change":"2023-12-16T21:43:19.275510+0000","last_active":"2023-12-16T21:43:21.368434+0000","last_peered":"2023-12-16T21:43:21.368434+0000","last_clean":"2023-12-16T21:43:21.368434+0000","last_became_active":"2023-12-16T21:43:19.275050+0000","last_became_peered":"2023-12-16T21:43:19.275050+0000","last_unstale":"2023-12-16T21:43:21.368434+0000","last_undegraded":"2023-12-16T21:43:21.368434+0000","last_fullsized":"2023-12-16T21:43:21.368434+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:40.032790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278011+0000","last_change":"2023-12-16T21:43:20.315034+0000","last_active":"2023-12-16T21:43:21.278011+0000","last_peered":"2023-12-16T21:43:21.278011+0000","last_clean":"2023-12-16T21:43:21.278011+0000","last_became_active":"2023-12-16T21:43:20.313238+0000","last_became_peered":"2023-12-16T21:43:20.313238+0000","last_unstale":"2023-12-16T21:43:21.278011+0000","last_undegraded":"2023-12-16T21:43:21.278011+0000","last_fullsized":"2023-12-16T21:43:21.278011+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:48:13.371146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367888+0000","last_change":"2023-12-16T21:43:19.280980+0000","last_active":"2023-12-16T21:43:21.367888+0000","last_peered":"2023-12-16T21:43:21.367888+0000","last_clean":"2023-12-16T21:43:21.367888+0000","last_became_active":"2023-12-16T21:43:19.280515+0000","last_became_peered":"2023-12-16T21:43:19.280515+0000","last_unstale":"2023-12-16T21:43:21.367888+0000","last_undegraded":"2023-12-16T21:43:21.367888+0000","last_fullsized":"2023-12-16T21:43:21.367888+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:25:01.268486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370619+0000","last_change":"2023-12-16T21:43:20.300148+0000","last_active":"2023-12-16T21:43:21.370619+0000","last_peered":"2023-12-16T21:43:21.370619+0000","last_clean":"2023-12-16T21:43:21.370619+0000","last_became_active":"2023-12-16T21:43:20.299228+0000","last_became_peered":"2023-12-16T21:43:20.299228+0000","last_unstale":"2023-12-16T21:43:21.370619+0000","last_undegraded":"2023-12-16T21:43:21.370619+0000","last_fullsized":"2023-12-16T21:43:21.370619+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:41.036474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367550+0000","last_change":"2023-12-16T21:43:19.283159+0000","last_active":"2023-12-16T21:43:21.367550+0000","last_peered":"2023-12-16T21:43:21.367550+0000","last_clean":"2023-12-16T21:43:21.367550+0000","last_became_active":"2023-12-16T21:43:19.281792+0000","last_became_peered":"2023-12-16T21:43:19.281792+0000","last_unstale":"2023-12-16T21:43:21.367550+0000","last_undegraded":"2023-12-16T21:43:21.367550+0000","last_fullsized":"2023-12-16T21:43:21.367550+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:49:42.347373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279852+0000","last_change":"2023-12-16T21:43:20.323769+0000","last_active":"2023-12-16T21:43:21.279852+0000","last_peered":"2023-12-16T21:43:21.279852+0000","last_clean":"2023-12-16T21:43:21.279852+0000","last_became_active":"2023-12-16T21:43:20.323193+0000","last_became_peered":"2023-12-16T21:43:20.323193+0000","last_unstale":"2023-12-16T21:43:21.279852+0000","last_undegraded":"2023-12-16T21:43:21.279852+0000","last_fullsized":"2023-12-16T21:43:21.279852+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:39:43.342628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370088+0000","last_change":"2023-12-16T21:43:19.287872+0000","last_active":"2023-12-16T21:43:21.370088+0000","last_peered":"2023-12-16T21:43:21.370088+0000","last_clean":"2023-12-16T21:43:21.370088+0000","last_became_active":"2023-12-16T21:43:19.282035+0000","last_became_peered":"2023-12-16T21:43:19.282035+0000","last_unstale":"2023-12-16T21:43:21.370088+0000","last_undegraded":"2023-12-16T21:43:21.370088+0000","last_fullsized":"2023-12-16T21:43:21.370088+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:38:53.822126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371355+0000","last_change":"2023-12-16T21:43:20.322915+0000","last_active":"2023-12-16T21:43:21.371355+0000","last_peered":"2023-12-16T21:43:21.371355+0000","last_clean":"2023-12-16T21:43:21.371355+0000","last_became_active":"2023-12-16T21:43:20.322239+0000","last_became_peered":"2023-12-16T21:43:20.322239+0000","last_unstale":"2023-12-16T21:43:21.371355+0000","last_undegraded":"2023-12-16T21:43:21.371355+0000","last_fullsized":"2023-12-16T21:43:21.371355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:32:57.024768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279798+0000","last_change":"2023-12-16T21:43:19.289609+0000","last_active":"2023-12-16T21:43:21.279798+0000","last_peered":"2023-12-16T21:43:21.279798+0000","last_clean":"2023-12-16T21:43:21.279798+0000","last_became_active":"2023-12-16T21:43:19.289091+0000","last_became_peered":"2023-12-16T21:43:19.289091+0000","last_unstale":"2023-12-16T21:43:21.279798+0000","last_undegraded":"2023-12-16T21:43:21.279798+0000","last_fullsized":"2023-12-16T21:43:21.279798+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:41:46.897269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368677+0000","last_change":"2023-12-16T21:43:20.310115+0000","last_active":"2023-12-16T21:43:21.368677+0000","last_peered":"2023-12-16T21:43:21.368677+0000","last_clean":"2023-12-16T21:43:21.368677+0000","last_became_active":"2023-12-16T21:43:20.309289+0000","last_became_peered":"2023-12-16T21:43:20.309289+0000","last_unstale":"2023-12-16T21:43:21.368677+0000","last_undegraded":"2023-12-16T21:43:21.368677+0000","last_fullsized":"2023-12-16T21:43:21.368677+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:23:07.200649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280119+0000","last_change":"2023-12-16T21:43:19.284529+0000","last_active":"2023-12-16T21:43:21.280119+0000","last_peered":"2023-12-16T21:43:21.280119+0000","last_clean":"2023-12-16T21:43:21.280119+0000","last_became_active":"2023-12-16T21:43:19.283815+0000","last_became_peered":"2023-12-16T21:43:19.283815+0000","last_unstale":"2023-12-16T21:43:21.280119+0000","last_undegraded":"2023-12-16T21:43:21.280119+0000","last_fullsized":"2023-12-16T21:43:21.280119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:03:02.840614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368061+0000","last_change":"2023-12-16T21:43:20.312410+0000","last_active":"2023-12-16T21:43:21.368061+0000","last_peered":"2023-12-16T21:43:21.368061+0000","last_clean":"2023-12-16T21:43:21.368061+0000","last_became_active":"2023-12-16T21:43:20.311742+0000","last_became_peered":"2023-12-16T21:43:20.311742+0000","last_unstale":"2023-12-16T21:43:21.368061+0000","last_undegraded":"2023-12-16T21:43:21.368061+0000","last_fullsized":"2023-12-16T21:43:21.368061+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:03:03.844297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280423+0000","last_change":"2023-12-16T21:43:19.284360+0000","last_active":"2023-12-16T21:43:21.280423+0000","last_peered":"2023-12-16T21:43:21.280423+0000","last_clean":"2023-12-16T21:43:21.280423+0000","last_became_active":"2023-12-16T21:43:19.283588+0000","last_became_peered":"2023-12-16T21:43:19.283588+0000","last_unstale":"2023-12-16T21:43:21.280423+0000","last_undegraded":"2023-12-16T21:43:21.280423+0000","last_fullsized":"2023-12-16T21:43:21.280423+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:58:34.582538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367537+0000","last_change":"2023-12-16T21:43:20.292017+0000","last_active":"2023-12-16T21:43:21.367537+0000","last_peered":"2023-12-16T21:43:21.367537+0000","last_clean":"2023-12-16T21:43:21.367537+0000","last_became_active":"2023-12-16T21:43:20.289647+0000","last_became_peered":"2023-12-16T21:43:20.289647+0000","last_unstale":"2023-12-16T21:43:21.367537+0000","last_undegraded":"2023-12-16T21:43:21.367537+0000","last_fullsized":"2023-12-16T21:43:21.367537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:58:35.586222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275585+0000","last_change":"2023-12-16T21:43:19.276671+0000","last_active":"2023-12-16T21:43:21.275585+0000","last_peered":"2023-12-16T21:43:21.275585+0000","last_clean":"2023-12-16T21:43:21.275585+0000","last_became_active":"2023-12-16T21:43:19.276346+0000","last_became_peered":"2023-12-16T21:43:19.276346+0000","last_unstale":"2023-12-16T21:43:21.275585+0000","last_undegraded":"2023-12-16T21:43:21.275585+0000","last_fullsized":"2023-12-16T21:43:21.275585+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:55:02.055360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367828+0000","last_change":"2023-12-16T21:43:20.302656+0000","last_active":"2023-12-16T21:43:21.367828+0000","last_peered":"2023-12-16T21:43:21.367828+0000","last_clean":"2023-12-16T21:43:21.367828+0000","last_became_active":"2023-12-16T21:43:20.301387+0000","last_became_peered":"2023-12-16T21:43:20.301387+0000","last_unstale":"2023-12-16T21:43:21.367828+0000","last_undegraded":"2023-12-16T21:43:21.367828+0000","last_fullsized":"2023-12-16T21:43:21.367828+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:41:47.900953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370403+0000","last_change":"2023-12-16T21:43:19.283687+0000","last_active":"2023-12-16T21:43:21.370403+0000","last_peered":"2023-12-16T21:43:21.370403+0000","last_clean":"2023-12-16T21:43:21.370403+0000","last_became_active":"2023-12-16T21:43:19.283020+0000","last_became_peered":"2023-12-16T21:43:19.283020+0000","last_unstale":"2023-12-16T21:43:21.370403+0000","last_undegraded":"2023-12-16T21:43:21.370403+0000","last_fullsized":"2023-12-16T21:43:21.370403+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:57:15.015059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367324+0000","last_change":"2023-12-16T21:43:20.300410+0000","last_active":"2023-12-16T21:43:21.367324+0000","last_peered":"2023-12-16T21:43:21.367324+0000","last_clean":"2023-12-16T21:43:21.367324+0000","last_became_active":"2023-12-16T21:43:20.299654+0000","last_became_peered":"2023-12-16T21:43:20.299654+0000","last_unstale":"2023-12-16T21:43:21.367324+0000","last_undegraded":"2023-12-16T21:43:21.367324+0000","last_fullsized":"2023-12-16T21:43:21.367324+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:12:11.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280064+0000","last_change":"2023-12-16T21:43:19.288868+0000","last_active":"2023-12-16T21:43:21.280064+0000","last_peered":"2023-12-16T21:43:21.280064+0000","last_clean":"2023-12-16T21:43:21.280064+0000","last_became_active":"2023-12-16T21:43:19.285277+0000","last_became_peered":"2023-12-16T21:43:19.285277+0000","last_unstale":"2023-12-16T21:43:21.280064+0000","last_undegraded":"2023-12-16T21:43:21.280064+0000","last_fullsized":"2023-12-16T21:43:21.280064+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:31:36.018521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277345+0000","last_change":"2023-12-16T21:43:20.307834+0000","last_active":"2023-12-16T21:43:21.277345+0000","last_peered":"2023-12-16T21:43:21.277345+0000","last_clean":"2023-12-16T21:43:21.277345+0000","last_became_active":"2023-12-16T21:43:20.307083+0000","last_became_peered":"2023-12-16T21:43:20.307083+0000","last_unstale":"2023-12-16T21:43:21.277345+0000","last_undegraded":"2023-12-16T21:43:21.277345+0000","last_fullsized":"2023-12-16T21:43:21.277345+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:16.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370375+0000","last_change":"2023-12-16T21:43:19.284519+0000","last_active":"2023-12-16T21:43:21.370375+0000","last_peered":"2023-12-16T21:43:21.370375+0000","last_clean":"2023-12-16T21:43:21.370375+0000","last_became_active":"2023-12-16T21:43:19.283895+0000","last_became_peered":"2023-12-16T21:43:19.283895+0000","last_unstale":"2023-12-16T21:43:21.370375+0000","last_undegraded":"2023-12-16T21:43:21.370375+0000","last_fullsized":"2023-12-16T21:43:21.370375+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:40.032790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368246+0000","last_change":"2023-12-16T21:43:20.321897+0000","last_active":"2023-12-16T21:43:21.368246+0000","last_peered":"2023-12-16T21:43:21.368246+0000","last_clean":"2023-12-16T21:43:21.368246+0000","last_became_active":"2023-12-16T21:43:20.321591+0000","last_became_peered":"2023-12-16T21:43:20.321591+0000","last_unstale":"2023-12-16T21:43:21.368246+0000","last_undegraded":"2023-12-16T21:43:21.368246+0000","last_fullsized":"2023-12-16T21:43:21.368246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:31:37.022205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370102+0000","last_change":"2023-12-16T21:43:19.279935+0000","last_active":"2023-12-16T21:43:21.370102+0000","last_peered":"2023-12-16T21:43:21.370102+0000","last_clean":"2023-12-16T21:43:21.370102+0000","last_became_active":"2023-12-16T21:43:19.279304+0000","last_became_peered":"2023-12-16T21:43:19.279304+0000","last_unstale":"2023-12-16T21:43:21.370102+0000","last_undegraded":"2023-12-16T21:43:21.370102+0000","last_fullsized":"2023-12-16T21:43:21.370102+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:39:42.338944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371666+0000","last_change":"2023-12-16T21:43:20.321762+0000","last_active":"2023-12-16T21:43:21.371666+0000","last_peered":"2023-12-16T21:43:21.371666+0000","last_clean":"2023-12-16T21:43:21.371666+0000","last_became_active":"2023-12-16T21:43:20.321157+0000","last_became_peered":"2023-12-16T21:43:20.321157+0000","last_unstale":"2023-12-16T21:43:21.371666+0000","last_undegraded":"2023-12-16T21:43:21.371666+0000","last_fullsized":"2023-12-16T21:43:21.371666+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:48:13.371146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275554+0000","last_change":"2023-12-16T21:43:19.271444+0000","last_active":"2023-12-16T21:43:21.275554+0000","last_peered":"2023-12-16T21:43:21.275554+0000","last_clean":"2023-12-16T21:43:21.275554+0000","last_became_active":"2023-12-16T21:43:19.271083+0000","last_became_peered":"2023-12-16T21:43:19.271083+0000","last_unstale":"2023-12-16T21:43:21.275554+0000","last_undegraded":"2023-12-16T21:43:21.275554+0000","last_fullsized":"2023-12-16T21:43:21.275554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:21:19.433920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275250+0000","last_change":"2023-12-16T21:43:20.322719+0000","last_active":"2023-12-16T21:43:21.275250+0000","last_peered":"2023-12-16T21:43:21.275250+0000","last_clean":"2023-12-16T21:43:21.275250+0000","last_became_active":"2023-12-16T21:43:20.322022+0000","last_became_peered":"2023-12-16T21:43:20.322022+0000","last_unstale":"2023-12-16T21:43:21.275250+0000","last_undegraded":"2023-12-16T21:43:21.275250+0000","last_fullsized":"2023-12-16T21:43:21.275250+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:43.887172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368801+0000","last_change":"2023-12-16T21:43:19.274044+0000","last_active":"2023-12-16T21:43:21.368801+0000","last_peered":"2023-12-16T21:43:21.368801+0000","last_clean":"2023-12-16T21:43:21.368801+0000","last_became_active":"2023-12-16T21:43:19.269096+0000","last_became_peered":"2023-12-16T21:43:19.269096+0000","last_unstale":"2023-12-16T21:43:21.368801+0000","last_undegraded":"2023-12-16T21:43:21.368801+0000","last_fullsized":"2023-12-16T21:43:21.368801+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:42.883488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370456+0000","last_change":"2023-12-16T21:43:20.305227+0000","last_active":"2023-12-16T21:43:21.370456+0000","last_peered":"2023-12-16T21:43:21.370456+0000","last_clean":"2023-12-16T21:43:21.370456+0000","last_became_active":"2023-12-16T21:43:20.304565+0000","last_became_peered":"2023-12-16T21:43:20.304565+0000","last_unstale":"2023-12-16T21:43:21.370456+0000","last_undegraded":"2023-12-16T21:43:21.370456+0000","last_fullsized":"2023-12-16T21:43:21.370456+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:17:02.836979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275683+0000","last_change":"2023-12-16T21:43:19.274011+0000","last_active":"2023-12-16T21:43:21.275683+0000","last_peered":"2023-12-16T21:43:21.275683+0000","last_clean":"2023-12-16T21:43:21.275683+0000","last_became_active":"2023-12-16T21:43:19.273273+0000","last_became_peered":"2023-12-16T21:43:19.273273+0000","last_unstale":"2023-12-16T21:43:21.275683+0000","last_undegraded":"2023-12-16T21:43:21.275683+0000","last_fullsized":"2023-12-16T21:43:21.275683+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:57:15.015059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275332+0000","last_change":"2023-12-16T21:43:20.306745+0000","last_active":"2023-12-16T21:43:21.275332+0000","last_peered":"2023-12-16T21:43:21.275332+0000","last_clean":"2023-12-16T21:43:21.275332+0000","last_became_active":"2023-12-16T21:43:20.305961+0000","last_became_peered":"2023-12-16T21:43:20.305961+0000","last_unstale":"2023-12-16T21:43:21.275332+0000","last_undegraded":"2023-12-16T21:43:21.275332+0000","last_fullsized":"2023-12-16T21:43:21.275332+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:38:54.825810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368516+0000","last_change":"2023-12-16T21:43:19.279580+0000","last_active":"2023-12-16T21:43:21.368516+0000","last_peered":"2023-12-16T21:43:21.368516+0000","last_clean":"2023-12-16T21:43:21.368516+0000","last_became_active":"2023-12-16T21:43:19.279134+0000","last_became_peered":"2023-12-16T21:43:19.279134+0000","last_unstale":"2023-12-16T21:43:21.368516+0000","last_undegraded":"2023-12-16T21:43:21.368516+0000","last_fullsized":"2023-12-16T21:43:21.368516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:38.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367417+0000","last_change":"2023-12-16T21:43:20.318172+0000","last_active":"2023-12-16T21:43:21.367417+0000","last_peered":"2023-12-16T21:43:21.367417+0000","last_clean":"2023-12-16T21:43:21.367417+0000","last_became_active":"2023-12-16T21:43:20.317616+0000","last_became_peered":"2023-12-16T21:43:20.317616+0000","last_unstale":"2023-12-16T21:43:21.367417+0000","last_undegraded":"2023-12-16T21:43:21.367417+0000","last_fullsized":"2023-12-16T21:43:21.367417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280293+0000","last_change":"2023-12-16T21:43:19.285249+0000","last_active":"2023-12-16T21:43:21.280293+0000","last_peered":"2023-12-16T21:43:21.280293+0000","last_clean":"2023-12-16T21:43:21.280293+0000","last_became_active":"2023-12-16T21:43:19.284732+0000","last_became_peered":"2023-12-16T21:43:19.284732+0000","last_unstale":"2023-12-16T21:43:21.280293+0000","last_undegraded":"2023-12-16T21:43:21.280293+0000","last_fullsized":"2023-12-16T21:43:21.280293+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:07:12.587135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279431+0000","last_change":"2023-12-16T21:43:20.311137+0000","last_active":"2023-12-16T21:43:21.279431+0000","last_peered":"2023-12-16T21:43:21.279431+0000","last_clean":"2023-12-16T21:43:21.279431+0000","last_became_active":"2023-12-16T21:43:20.310423+0000","last_became_peered":"2023-12-16T21:43:20.310423+0000","last_unstale":"2023-12-16T21:43:21.279431+0000","last_undegraded":"2023-12-16T21:43:21.279431+0000","last_fullsized":"2023-12-16T21:43:21.279431+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:39.005796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367667+0000","last_change":"2023-12-16T21:43:19.287996+0000","last_active":"2023-12-16T21:43:21.367667+0000","last_peered":"2023-12-16T21:43:21.367667+0000","last_clean":"2023-12-16T21:43:21.367667+0000","last_became_active":"2023-12-16T21:43:19.286719+0000","last_became_peered":"2023-12-16T21:43:19.286719+0000","last_unstale":"2023-12-16T21:43:21.367667+0000","last_undegraded":"2023-12-16T21:43:21.367667+0000","last_fullsized":"2023-12-16T21:43:21.367667+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:32:36.251240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277701+0000","last_change":"2023-12-16T21:43:20.298745+0000","last_active":"2023-12-16T21:43:21.277701+0000","last_peered":"2023-12-16T21:43:21.277701+0000","last_clean":"2023-12-16T21:43:21.277701+0000","last_became_active":"2023-12-16T21:43:20.298258+0000","last_became_peered":"2023-12-16T21:43:20.298258+0000","last_unstale":"2023-12-16T21:43:21.277701+0000","last_undegraded":"2023-12-16T21:43:21.277701+0000","last_fullsized":"2023-12-16T21:43:21.277701+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:32:57.024768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370116+0000","last_change":"2023-12-16T21:43:19.293689+0000","last_active":"2023-12-16T21:43:21.370116+0000","last_peered":"2023-12-16T21:43:21.370116+0000","last_clean":"2023-12-16T21:43:21.370116+0000","last_became_active":"2023-12-16T21:43:19.293025+0000","last_became_peered":"2023-12-16T21:43:19.293025+0000","last_unstale":"2023-12-16T21:43:21.370116+0000","last_undegraded":"2023-12-16T21:43:21.370116+0000","last_fullsized":"2023-12-16T21:43:21.370116+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:27:20.836500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-16T21:43:20.286524+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:20.286524+0000","last_peered":"2023-12-16T21:43:20.286524+0000","last_clean":"2023-12-16T21:43:20.286524+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:20.286524+0000","last_undegraded":"2023-12-16T21:43:20.286524+0000","last_fullsized":"2023-12-16T21:43:20.286524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":719117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":129275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":129331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":128763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":352256,"data_stored":130293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":339968,"data_stored":128729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":352256,"data_stored":130293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":929792,"data_stored":719027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:27.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.947+0000 7f483b989640 1 Processor -- start 2023-12-16T21:43:27.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.947+0000 7f483b989640 1 -- start start 2023-12-16T21:43:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f483b989640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48340af2e0 0x7f4834147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f483b989640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340af770 con 0x7f48340ad7c0 2023-12-16T21:43:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f483b989640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48340aeea0 con 0x7f483406c920 2023-12-16T21:43:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f483b989640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48340afc20 con 0x7f48340af2e0 2023-12-16T21:43:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4839f26640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f48340ad7c0 0x7f483414def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:42470/0 (socket says 172.21.15.46:42470) 2023-12-16T21:43:27.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4838f24640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f48340af2e0 0x7f4834147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4839f26640 1 -- 172.21.15.46:0/2064719120 learned_addr learned my addr 172.21.15.46:0/2064719120 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4833fff640 1 -- 172.21.15.46:0/2064719120 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394122631 0 0) 0x7f48340af770 con 0x7f48340ad7c0 2023-12-16T21:43:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4833fff640 1 -- 172.21.15.46:0/2064719120 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f481c003530 con 0x7f48340ad7c0 2023-12-16T21:43:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4838f24640 1 -- 172.21.15.46:0/2064719120 >> v1:172.21.15.161:6789/0 conn(0x7f483406c920 legacy=0x7f48340ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:27.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4838f24640 1 -- 172.21.15.46:0/2064719120 >> v1:172.21.15.46:6789/0 conn(0x7f48340ad7c0 legacy=0x7f483414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:27.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.951+0000 7f4838f24640 1 -- 172.21.15.46:0/2064719120 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483406cd00 con 0x7f48340af2e0 2023-12-16T21:43:27.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f4838f24640 1 --2- 172.21.15.46:0/2064719120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340af2e0 0x7f4834147080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f482801c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7def19b541471863 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f4833fff640 1 -- 172.21.15.46:0/2064719120 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48280a58e0 con 0x7f48340af2e0 2023-12-16T21:43:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f4833fff640 1 -- 172.21.15.46:0/2064719120 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48280a5a40 con 0x7f48340af2e0 2023-12-16T21:43:27.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f4833fff640 1 -- 172.21.15.46:0/2064719120 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48280a5db0 con 0x7f48340af2e0 2023-12-16T21:43:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- 172.21.15.46:0/2064719120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340af2e0 msgr2=0x7f4834147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 --2- 172.21.15.46:0/2064719120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340af2e0 0x7f4834147080 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4828009d90 tx=0x7f482801c800 comp rx=0 tx=0).stop 2023-12-16T21:43:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- 172.21.15.46:0/2064719120 shutdown_connections 2023-12-16T21:43:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 --2- 172.21.15.46:0/2064719120 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340af2e0 0x7f4834147080 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- 172.21.15.46:0/2064719120 >> 172.21.15.46:0/2064719120 conn(0x7f48340b48e0 msgr2=0x7f48340b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:27.959 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- 172.21.15.46:0/2064719120 shutdown_connections 2023-12-16T21:43:27.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- 172.21.15.46:0/2064719120 wait complete. 2023-12-16T21:43:27.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 Processor -- start 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- start start 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f483406c920 0x7f483413ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340ad7c0 0x7f483413c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48341e02c0 0x7f4834146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834146b50 con 0x7f48340ad7c0 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4834146180 con 0x7f48341e02c0 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f483b989640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483413caa0 con 0x7f483406c920 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.955+0000 7f4838f24640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340ad7c0 0x7f483413c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340ad7c0 0x7f483413c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53908/0 (socket says 172.21.15.46:53908) 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 -- 172.21.15.46:0/2907246735 learned_addr learned my addr 172.21.15.46:0/2907246735 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4839725640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f483406c920 0x7f483413ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4839f26640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48341e02c0 0x7f4834146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:27.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 -- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f483406c920 msgr2=0x7f483413ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f483406c920 0x7f483413ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 -- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48341e02c0 msgr2=0x7f4834146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48341e02c0 0x7f4834146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 -- 172.21.15.46:0/2907246735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4828009a70 con 0x7f48340ad7c0 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4838f24640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340ad7c0 0x7f483413c390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f482801cce0 tx=0x7f48280195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4831ffb640 1 -- 172.21.15.46:0/2907246735 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48280ad850 con 0x7f48340ad7c0 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4831ffb640 1 -- 172.21.15.46:0/2907246735 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48280ad9b0 con 0x7f48340ad7c0 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483413da50 con 0x7f48340ad7c0 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4831ffb640 1 -- 172.21.15.46:0/2907246735 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48280add60 con 0x7f48340ad7c0 2023-12-16T21:43:27.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483413f9e0 con 0x7f48340ad7c0 2023-12-16T21:43:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48340679e0 con 0x7f48340ad7c0 2023-12-16T21:43:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4831ffb640 1 -- 172.21.15.46:0/2907246735 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f48280b4070 con 0x7f48340ad7c0 2023-12-16T21:43:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4831ffb640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48200664a0 0x7f4820068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4831ffb640 1 -- 172.21.15.46:0/2907246735 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f482811e630 con 0x7f48340ad7c0 2023-12-16T21:43:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4839725640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48200664a0 0x7f4820068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:27.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.959+0000 7f4839725640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48200664a0 0x7f4820068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4824000f70 tx=0x7f4824009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:27.963+0000 7f4831ffb640 1 -- 172.21.15.46:0/2907246735 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f48280ebb40 con 0x7f48340ad7c0 2023-12-16T21:43:28.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.119+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f483414f050 con 0x7f48200664a0 2023-12-16T21:43:28.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.139+0000 7f4831ffb640 1 -- 172.21.15.46:0/2907246735 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264659 (secure 0 0 0) 0x7f483414f050 con 0x7f48200664a0 2023-12-16T21:43:28.146 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:28.148 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48200664a0 msgr2=0x7f4820068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.149 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48200664a0 0x7f4820068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4824000f70 tx=0x7f4824009040 comp rx=0 tx=0).stop 2023-12-16T21:43:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340ad7c0 msgr2=0x7f483413c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340ad7c0 0x7f483413c390 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f482801cce0 tx=0x7f48280195a0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 shutdown_connections 2023-12-16T21:43:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f48200664a0 0x7f4820068960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f48341e02c0 0x7f4834146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.150 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48340ad7c0 0x7f483413c390 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 --2- 172.21.15.46:0/2907246735 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f483406c920 0x7f483413ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 >> 172.21.15.46:0/2907246735 conn(0x7f48340b48e0 msgr2=0x7f483406be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 shutdown_connections 2023-12-16T21:43:28.151 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.143+0000 7f483b989640 1 -- 172.21.15.46:0/2907246735 wait complete. 2023-12-16T21:43:28.164 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2023-12-16T21:43:26.365910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":136,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236688,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738032,"statfs":{"total":773094113280,"available":772851744768,"internally_reserved":0,"allocated":3940352,"data_stored":2214828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.117226"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370766+0000","last_change":"2023-12-16T21:43:20.325568+0000","last_active":"2023-12-16T21:43:21.370766+0000","last_peered":"2023-12-16T21:43:21.370766+0000","last_clean":"2023-12-16T21:43:21.370766+0000","last_became_active":"2023-12-16T21:43:20.324943+0000","last_became_peered":"2023-12-16T21:43:20.324943+0000","last_unstale":"2023-12-16T21:43:21.370766+0000","last_undegraded":"2023-12-16T21:43:21.370766+0000","last_fullsized":"2023-12-16T21:43:21.370766+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:39.005796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369403+0000","last_change":"2023-12-16T21:43:20.305510+0000","last_active":"2023-12-16T21:43:21.369403+0000","last_peered":"2023-12-16T21:43:21.369403+0000","last_clean":"2023-12-16T21:43:21.369403+0000","last_became_active":"2023-12-16T21:43:20.304590+0000","last_became_peered":"2023-12-16T21:43:20.304590+0000","last_unstale":"2023-12-16T21:43:21.369403+0000","last_undegraded":"2023-12-16T21:43:21.369403+0000","last_fullsized":"2023-12-16T21:43:21.369403+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:32:36.029124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370894+0000","last_change":"2023-12-16T21:43:20.322443+0000","last_active":"2023-12-16T21:43:21.370894+0000","last_peered":"2023-12-16T21:43:21.370894+0000","last_clean":"2023-12-16T21:43:21.370894+0000","last_became_active":"2023-12-16T21:43:20.321782+0000","last_became_peered":"2023-12-16T21:43:20.321782+0000","last_unstale":"2023-12-16T21:43:21.370894+0000","last_undegraded":"2023-12-16T21:43:21.370894+0000","last_fullsized":"2023-12-16T21:43:21.370894+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368640+0000","last_change":"2023-12-16T21:43:20.318503+0000","last_active":"2023-12-16T21:43:21.368640+0000","last_peered":"2023-12-16T21:43:21.368640+0000","last_clean":"2023-12-16T21:43:21.368640+0000","last_became_active":"2023-12-16T21:43:20.317848+0000","last_became_peered":"2023-12-16T21:43:20.317848+0000","last_unstale":"2023-12-16T21:43:21.368640+0000","last_undegraded":"2023-12-16T21:43:21.368640+0000","last_fullsized":"2023-12-16T21:43:21.368640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:32:37.254923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367453+0000","last_change":"2023-12-16T21:43:20.313033+0000","last_active":"2023-12-16T21:43:21.367453+0000","last_peered":"2023-12-16T21:43:21.367453+0000","last_clean":"2023-12-16T21:43:21.367453+0000","last_became_active":"2023-12-16T21:43:20.311438+0000","last_became_peered":"2023-12-16T21:43:20.311438+0000","last_unstale":"2023-12-16T21:43:21.367453+0000","last_undegraded":"2023-12-16T21:43:21.367453+0000","last_fullsized":"2023-12-16T21:43:21.367453+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:49:33.889831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368240+0000","last_change":"2023-12-16T21:43:20.325278+0000","last_active":"2023-12-16T21:43:21.368240+0000","last_peered":"2023-12-16T21:43:21.368240+0000","last_clean":"2023-12-16T21:43:21.368240+0000","last_became_active":"2023-12-16T21:43:20.324935+0000","last_became_peered":"2023-12-16T21:43:20.324935+0000","last_unstale":"2023-12-16T21:43:21.368240+0000","last_undegraded":"2023-12-16T21:43:21.368240+0000","last_fullsized":"2023-12-16T21:43:21.368240+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:15:29.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278397+0000","last_change":"2023-12-16T21:43:20.300989+0000","last_active":"2023-12-16T21:43:21.278397+0000","last_peered":"2023-12-16T21:43:21.278397+0000","last_clean":"2023-12-16T21:43:21.278397+0000","last_became_active":"2023-12-16T21:43:20.300268+0000","last_became_peered":"2023-12-16T21:43:20.300268+0000","last_unstale":"2023-12-16T21:43:21.278397+0000","last_undegraded":"2023-12-16T21:43:21.278397+0000","last_fullsized":"2023-12-16T21:43:21.278397+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:46:42.032255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277462+0000","last_change":"2023-12-16T21:43:20.299012+0000","last_active":"2023-12-16T21:43:21.277462+0000","last_peered":"2023-12-16T21:43:21.277462+0000","last_clean":"2023-12-16T21:43:21.277462+0000","last_became_active":"2023-12-16T21:43:20.298084+0000","last_became_peered":"2023-12-16T21:43:20.298084+0000","last_unstale":"2023-12-16T21:43:21.277462+0000","last_undegraded":"2023-12-16T21:43:21.277462+0000","last_fullsized":"2023-12-16T21:43:21.277462+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280405+0000","last_change":"2023-12-16T21:43:20.318994+0000","last_active":"2023-12-16T21:43:21.280405+0000","last_peered":"2023-12-16T21:43:21.280405+0000","last_clean":"2023-12-16T21:43:21.280405+0000","last_became_active":"2023-12-16T21:43:20.318279+0000","last_became_peered":"2023-12-16T21:43:20.318279+0000","last_unstale":"2023-12-16T21:43:21.280405+0000","last_undegraded":"2023-12-16T21:43:21.280405+0000","last_fullsized":"2023-12-16T21:43:21.280405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:32:37.254923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369410+0000","last_change":"2023-12-16T21:43:20.317059+0000","last_active":"2023-12-16T21:43:21.369410+0000","last_peered":"2023-12-16T21:43:21.369410+0000","last_clean":"2023-12-16T21:43:21.369410+0000","last_became_active":"2023-12-16T21:43:20.316318+0000","last_became_peered":"2023-12-16T21:43:20.316318+0000","last_unstale":"2023-12-16T21:43:21.369410+0000","last_undegraded":"2023-12-16T21:43:21.369410+0000","last_fullsized":"2023-12-16T21:43:21.369410+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:16.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278127+0000","last_change":"2023-12-16T21:43:20.328830+0000","last_active":"2023-12-16T21:43:21.278127+0000","last_peered":"2023-12-16T21:43:21.278127+0000","last_clean":"2023-12-16T21:43:21.278127+0000","last_became_active":"2023-12-16T21:43:20.328205+0000","last_became_peered":"2023-12-16T21:43:20.328205+0000","last_unstale":"2023-12-16T21:43:21.278127+0000","last_undegraded":"2023-12-16T21:43:21.278127+0000","last_fullsized":"2023-12-16T21:43:21.278127+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:32:36.029124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371475+0000","last_change":"2023-12-16T21:43:20.323210+0000","last_active":"2023-12-16T21:43:21.371475+0000","last_peered":"2023-12-16T21:43:21.371475+0000","last_clean":"2023-12-16T21:43:21.371475+0000","last_became_active":"2023-12-16T21:43:20.320416+0000","last_became_peered":"2023-12-16T21:43:20.320416+0000","last_unstale":"2023-12-16T21:43:21.371475+0000","last_undegraded":"2023-12-16T21:43:21.371475+0000","last_fullsized":"2023-12-16T21:43:21.371475+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:21:43.837309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367640+0000","last_change":"2023-12-16T21:43:20.317694+0000","last_active":"2023-12-16T21:43:21.367640+0000","last_peered":"2023-12-16T21:43:21.367640+0000","last_clean":"2023-12-16T21:43:21.367640+0000","last_became_active":"2023-12-16T21:43:20.316996+0000","last_became_peered":"2023-12-16T21:43:20.316996+0000","last_unstale":"2023-12-16T21:43:21.367640+0000","last_undegraded":"2023-12-16T21:43:21.367640+0000","last_fullsized":"2023-12-16T21:43:21.367640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:00:23.034732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368436+0000","last_change":"2023-12-16T21:43:20.304950+0000","last_active":"2023-12-16T21:43:21.368436+0000","last_peered":"2023-12-16T21:43:21.368436+0000","last_clean":"2023-12-16T21:43:21.368436+0000","last_became_active":"2023-12-16T21:43:20.304097+0000","last_became_peered":"2023-12-16T21:43:20.304097+0000","last_unstale":"2023-12-16T21:43:21.368436+0000","last_undegraded":"2023-12-16T21:43:21.368436+0000","last_fullsized":"2023-12-16T21:43:21.368436+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:15:32.234255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370718+0000","last_change":"2023-12-16T21:43:20.319973+0000","last_active":"2023-12-16T21:43:21.370718+0000","last_peered":"2023-12-16T21:43:21.370718+0000","last_clean":"2023-12-16T21:43:21.370718+0000","last_became_active":"2023-12-16T21:43:20.319308+0000","last_became_peered":"2023-12-16T21:43:20.319308+0000","last_unstale":"2023-12-16T21:43:21.370718+0000","last_undegraded":"2023-12-16T21:43:21.370718+0000","last_fullsized":"2023-12-16T21:43:21.370718+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:16.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278182+0000","last_change":"2023-12-16T21:43:20.318376+0000","last_active":"2023-12-16T21:43:21.278182+0000","last_peered":"2023-12-16T21:43:21.278182+0000","last_clean":"2023-12-16T21:43:21.278182+0000","last_became_active":"2023-12-16T21:43:20.317573+0000","last_became_peered":"2023-12-16T21:43:20.317573+0000","last_unstale":"2023-12-16T21:43:21.278182+0000","last_undegraded":"2023-12-16T21:43:21.278182+0000","last_fullsized":"2023-12-16T21:43:21.278182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:39:10.806527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278890+0000","last_change":"2023-12-16T21:43:20.311318+0000","last_active":"2023-12-16T21:43:21.278890+0000","last_peered":"2023-12-16T21:43:21.278890+0000","last_clean":"2023-12-16T21:43:21.278890+0000","last_became_active":"2023-12-16T21:43:20.310591+0000","last_became_peered":"2023-12-16T21:43:20.310591+0000","last_unstale":"2023-12-16T21:43:21.278890+0000","last_undegraded":"2023-12-16T21:43:21.278890+0000","last_fullsized":"2023-12-16T21:43:21.278890+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:52:20.633557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369493+0000","last_change":"2023-12-16T21:43:20.321144+0000","last_active":"2023-12-16T21:43:21.369493+0000","last_peered":"2023-12-16T21:43:21.369493+0000","last_clean":"2023-12-16T21:43:21.369493+0000","last_became_active":"2023-12-16T21:43:20.320838+0000","last_became_peered":"2023-12-16T21:43:20.320838+0000","last_unstale":"2023-12-16T21:43:21.369493+0000","last_undegraded":"2023-12-16T21:43:21.369493+0000","last_fullsized":"2023-12-16T21:43:21.369493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:39:10.806527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369713+0000","last_change":"2023-12-16T21:43:20.327347+0000","last_active":"2023-12-16T21:43:21.369713+0000","last_peered":"2023-12-16T21:43:21.369713+0000","last_clean":"2023-12-16T21:43:21.369713+0000","last_became_active":"2023-12-16T21:43:20.326953+0000","last_became_peered":"2023-12-16T21:43:20.326953+0000","last_unstale":"2023-12-16T21:43:21.369713+0000","last_undegraded":"2023-12-16T21:43:21.369713+0000","last_fullsized":"2023-12-16T21:43:21.369713+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:52:20.633557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367790+0000","last_change":"2023-12-16T21:43:20.324821+0000","last_active":"2023-12-16T21:43:21.367790+0000","last_peered":"2023-12-16T21:43:21.367790+0000","last_clean":"2023-12-16T21:43:21.367790+0000","last_became_active":"2023-12-16T21:43:20.324069+0000","last_became_peered":"2023-12-16T21:43:20.324069+0000","last_unstale":"2023-12-16T21:43:21.367790+0000","last_undegraded":"2023-12-16T21:43:21.367790+0000","last_fullsized":"2023-12-16T21:43:21.367790+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:00:46.209082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279491+0000","last_change":"2023-12-16T21:43:20.302529+0000","last_active":"2023-12-16T21:43:21.279491+0000","last_peered":"2023-12-16T21:43:21.279491+0000","last_clean":"2023-12-16T21:43:21.279491+0000","last_became_active":"2023-12-16T21:43:20.302091+0000","last_became_peered":"2023-12-16T21:43:20.302091+0000","last_unstale":"2023-12-16T21:43:21.279491+0000","last_undegraded":"2023-12-16T21:43:21.279491+0000","last_fullsized":"2023-12-16T21:43:21.279491+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:43.887172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279873+0000","last_change":"2023-12-16T21:43:20.302223+0000","last_active":"2023-12-16T21:43:21.279873+0000","last_peered":"2023-12-16T21:43:21.279873+0000","last_clean":"2023-12-16T21:43:21.279873+0000","last_became_active":"2023-12-16T21:43:20.301339+0000","last_became_peered":"2023-12-16T21:43:20.301339+0000","last_unstale":"2023-12-16T21:43:21.279873+0000","last_undegraded":"2023-12-16T21:43:21.279873+0000","last_fullsized":"2023-12-16T21:43:21.279873+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:49:43.351056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.274882+0000","last_change":"2023-12-16T21:43:20.314553+0000","last_active":"2023-12-16T21:43:21.274882+0000","last_peered":"2023-12-16T21:43:21.274882+0000","last_clean":"2023-12-16T21:43:21.274882+0000","last_became_active":"2023-12-16T21:43:20.313906+0000","last_became_peered":"2023-12-16T21:43:20.313906+0000","last_unstale":"2023-12-16T21:43:21.274882+0000","last_undegraded":"2023-12-16T21:43:21.274882+0000","last_fullsized":"2023-12-16T21:43:21.274882+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:41.036474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370150+0000","last_change":"2023-12-16T21:43:20.319665+0000","last_active":"2023-12-16T21:43:21.370150+0000","last_peered":"2023-12-16T21:43:21.370150+0000","last_clean":"2023-12-16T21:43:21.370150+0000","last_became_active":"2023-12-16T21:43:20.319110+0000","last_became_peered":"2023-12-16T21:43:20.319110+0000","last_unstale":"2023-12-16T21:43:21.370150+0000","last_undegraded":"2023-12-16T21:43:21.370150+0000","last_fullsized":"2023-12-16T21:43:21.370150+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:25:02.272170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369993+0000","last_change":"2023-12-16T21:43:20.317707+0000","last_active":"2023-12-16T21:43:21.369993+0000","last_peered":"2023-12-16T21:43:21.369993+0000","last_clean":"2023-12-16T21:43:21.369993+0000","last_became_active":"2023-12-16T21:43:20.317112+0000","last_became_peered":"2023-12-16T21:43:20.317112+0000","last_unstale":"2023-12-16T21:43:21.369993+0000","last_undegraded":"2023-12-16T21:43:21.369993+0000","last_fullsized":"2023-12-16T21:43:21.369993+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369583+0000","last_change":"2023-12-16T21:43:20.312988+0000","last_active":"2023-12-16T21:43:21.369583+0000","last_peered":"2023-12-16T21:43:21.369583+0000","last_clean":"2023-12-16T21:43:21.369583+0000","last_became_active":"2023-12-16T21:43:20.312213+0000","last_became_peered":"2023-12-16T21:43:20.312213+0000","last_unstale":"2023-12-16T21:43:21.369583+0000","last_undegraded":"2023-12-16T21:43:21.369583+0000","last_fullsized":"2023-12-16T21:43:21.369583+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:46:42.032255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369166+0000","last_change":"2023-12-16T21:43:20.317515+0000","last_active":"2023-12-16T21:43:21.369166+0000","last_peered":"2023-12-16T21:43:21.369166+0000","last_clean":"2023-12-16T21:43:21.369166+0000","last_became_active":"2023-12-16T21:43:20.316614+0000","last_became_peered":"2023-12-16T21:43:20.316614+0000","last_unstale":"2023-12-16T21:43:21.369166+0000","last_undegraded":"2023-12-16T21:43:21.369166+0000","last_fullsized":"2023-12-16T21:43:21.369166+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:25:02.272170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371832+0000","last_change":"2023-12-16T21:43:20.319882+0000","last_active":"2023-12-16T21:43:21.371832+0000","last_peered":"2023-12-16T21:43:21.371832+0000","last_clean":"2023-12-16T21:43:21.371832+0000","last_became_active":"2023-12-16T21:43:20.319160+0000","last_became_peered":"2023-12-16T21:43:20.319160+0000","last_unstale":"2023-12-16T21:43:21.371832+0000","last_undegraded":"2023-12-16T21:43:21.371832+0000","last_fullsized":"2023-12-16T21:43:21.371832+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:32:36.029124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279521+0000","last_change":"2023-12-16T21:43:20.317629+0000","last_active":"2023-12-16T21:43:21.279521+0000","last_peered":"2023-12-16T21:43:21.279521+0000","last_clean":"2023-12-16T21:43:21.279521+0000","last_became_active":"2023-12-16T21:43:20.316933+0000","last_became_peered":"2023-12-16T21:43:20.316933+0000","last_unstale":"2023-12-16T21:43:21.279521+0000","last_undegraded":"2023-12-16T21:43:21.279521+0000","last_fullsized":"2023-12-16T21:43:21.279521+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:38:54.825810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277539+0000","last_change":"2023-12-16T21:43:20.310554+0000","last_active":"2023-12-16T21:43:21.277539+0000","last_peered":"2023-12-16T21:43:21.277539+0000","last_clean":"2023-12-16T21:43:21.277539+0000","last_became_active":"2023-12-16T21:43:20.310080+0000","last_became_peered":"2023-12-16T21:43:20.310080+0000","last_unstale":"2023-12-16T21:43:21.277539+0000","last_undegraded":"2023-12-16T21:43:21.277539+0000","last_fullsized":"2023-12-16T21:43:21.277539+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:15:29.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370263+0000","last_change":"2023-12-16T21:43:20.305440+0000","last_active":"2023-12-16T21:43:21.370263+0000","last_peered":"2023-12-16T21:43:21.370263+0000","last_clean":"2023-12-16T21:43:21.370263+0000","last_became_active":"2023-12-16T21:43:20.304740+0000","last_became_peered":"2023-12-16T21:43:20.304740+0000","last_unstale":"2023-12-16T21:43:21.370263+0000","last_undegraded":"2023-12-16T21:43:21.370263+0000","last_fullsized":"2023-12-16T21:43:21.370263+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:21:20.437604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277815+0000","last_change":"2023-12-16T21:43:20.306619+0000","last_active":"2023-12-16T21:43:21.277815+0000","last_peered":"2023-12-16T21:43:21.277815+0000","last_clean":"2023-12-16T21:43:21.277815+0000","last_became_active":"2023-12-16T21:43:20.305996+0000","last_became_peered":"2023-12-16T21:43:20.305996+0000","last_unstale":"2023-12-16T21:43:21.277815+0000","last_undegraded":"2023-12-16T21:43:21.277815+0000","last_fullsized":"2023-12-16T21:43:21.277815+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:15:32.234255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275176+0000","last_change":"2023-12-16T21:43:20.311721+0000","last_active":"2023-12-16T21:43:21.275176+0000","last_peered":"2023-12-16T21:43:21.275176+0000","last_clean":"2023-12-16T21:43:21.275176+0000","last_became_active":"2023-12-16T21:43:20.311029+0000","last_became_peered":"2023-12-16T21:43:20.311029+0000","last_unstale":"2023-12-16T21:43:21.275176+0000","last_undegraded":"2023-12-16T21:43:21.275176+0000","last_fullsized":"2023-12-16T21:43:21.275176+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:39.005796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371050+0000","last_change":"2023-12-16T21:43:19.288334+0000","last_active":"2023-12-16T21:43:21.371050+0000","last_peered":"2023-12-16T21:43:21.371050+0000","last_clean":"2023-12-16T21:43:21.371050+0000","last_became_active":"2023-12-16T21:43:19.287636+0000","last_became_peered":"2023-12-16T21:43:19.287636+0000","last_unstale":"2023-12-16T21:43:21.371050+0000","last_undegraded":"2023-12-16T21:43:21.371050+0000","last_fullsized":"2023-12-16T21:43:21.371050+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:23:17.386717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370936+0000","last_change":"2023-12-16T21:43:20.310784+0000","last_active":"2023-12-16T21:43:21.370936+0000","last_peered":"2023-12-16T21:43:21.370936+0000","last_clean":"2023-12-16T21:43:21.370936+0000","last_became_active":"2023-12-16T21:43:20.310062+0000","last_became_peered":"2023-12-16T21:43:20.310062+0000","last_unstale":"2023-12-16T21:43:21.370936+0000","last_undegraded":"2023-12-16T21:43:21.370936+0000","last_fullsized":"2023-12-16T21:43:21.370936+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:43.887172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275455+0000","last_change":"2023-12-16T21:43:19.276377+0000","last_active":"2023-12-16T21:43:21.275455+0000","last_peered":"2023-12-16T21:43:21.275455+0000","last_clean":"2023-12-16T21:43:21.275455+0000","last_became_active":"2023-12-16T21:43:19.275793+0000","last_became_peered":"2023-12-16T21:43:19.275793+0000","last_unstale":"2023-12-16T21:43:21.275455+0000","last_undegraded":"2023-12-16T21:43:21.275455+0000","last_fullsized":"2023-12-16T21:43:21.275455+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:27:20.836500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280256+0000","last_change":"2023-12-16T21:43:20.316280+0000","last_active":"2023-12-16T21:43:21.280256+0000","last_peered":"2023-12-16T21:43:21.280256+0000","last_clean":"2023-12-16T21:43:21.280256+0000","last_became_active":"2023-12-16T21:43:20.315797+0000","last_became_peered":"2023-12-16T21:43:20.315797+0000","last_unstale":"2023-12-16T21:43:21.280256+0000","last_undegraded":"2023-12-16T21:43:21.280256+0000","last_fullsized":"2023-12-16T21:43:21.280256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:00:19.396579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369788+0000","last_change":"2023-12-16T21:43:19.279704+0000","last_active":"2023-12-16T21:43:21.369788+0000","last_peered":"2023-12-16T21:43:21.369788+0000","last_clean":"2023-12-16T21:43:21.369788+0000","last_became_active":"2023-12-16T21:43:19.279378+0000","last_became_peered":"2023-12-16T21:43:19.279378+0000","last_unstale":"2023-12-16T21:43:21.369788+0000","last_undegraded":"2023-12-16T21:43:21.369788+0000","last_fullsized":"2023-12-16T21:43:21.369788+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:40:30.337768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369255+0000","last_change":"2023-12-16T21:43:20.318988+0000","last_active":"2023-12-16T21:43:21.369255+0000","last_peered":"2023-12-16T21:43:21.369255+0000","last_clean":"2023-12-16T21:43:21.369255+0000","last_became_active":"2023-12-16T21:43:20.318381+0000","last_became_peered":"2023-12-16T21:43:20.318381+0000","last_unstale":"2023-12-16T21:43:21.369255+0000","last_undegraded":"2023-12-16T21:43:21.369255+0000","last_fullsized":"2023-12-16T21:43:21.369255+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:40:31.341452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367919+0000","last_change":"2023-12-16T21:43:19.267679+0000","last_active":"2023-12-16T21:43:21.367919+0000","last_peered":"2023-12-16T21:43:21.367919+0000","last_clean":"2023-12-16T21:43:21.367919+0000","last_became_active":"2023-12-16T21:43:19.267292+0000","last_became_peered":"2023-12-16T21:43:19.267292+0000","last_unstale":"2023-12-16T21:43:21.367919+0000","last_undegraded":"2023-12-16T21:43:21.367919+0000","last_fullsized":"2023-12-16T21:43:21.367919+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:51:27.921297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277959+0000","last_change":"2023-12-16T21:43:20.311672+0000","last_active":"2023-12-16T21:43:21.277959+0000","last_peered":"2023-12-16T21:43:21.277959+0000","last_clean":"2023-12-16T21:43:21.277959+0000","last_became_active":"2023-12-16T21:43:20.310932+0000","last_became_peered":"2023-12-16T21:43:20.310932+0000","last_unstale":"2023-12-16T21:43:21.277959+0000","last_undegraded":"2023-12-16T21:43:21.277959+0000","last_fullsized":"2023-12-16T21:43:21.277959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:21:43.837309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279750+0000","last_change":"2023-12-16T21:43:19.288807+0000","last_active":"2023-12-16T21:43:21.279750+0000","last_peered":"2023-12-16T21:43:21.279750+0000","last_clean":"2023-12-16T21:43:21.279750+0000","last_became_active":"2023-12-16T21:43:19.284430+0000","last_became_peered":"2023-12-16T21:43:19.284430+0000","last_unstale":"2023-12-16T21:43:21.279750+0000","last_undegraded":"2023-12-16T21:43:21.279750+0000","last_fullsized":"2023-12-16T21:43:21.279750+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:23:17.386717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371439+0000","last_change":"2023-12-16T21:43:20.319794+0000","last_active":"2023-12-16T21:43:21.371439+0000","last_peered":"2023-12-16T21:43:21.371439+0000","last_clean":"2023-12-16T21:43:21.371439+0000","last_became_active":"2023-12-16T21:43:20.319191+0000","last_became_peered":"2023-12-16T21:43:20.319191+0000","last_unstale":"2023-12-16T21:43:21.371439+0000","last_undegraded":"2023-12-16T21:43:21.371439+0000","last_fullsized":"2023-12-16T21:43:21.371439+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:15:32.234255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279764+0000","last_change":"2023-12-16T21:43:19.283093+0000","last_active":"2023-12-16T21:43:21.279764+0000","last_peered":"2023-12-16T21:43:21.279764+0000","last_clean":"2023-12-16T21:43:21.279764+0000","last_became_active":"2023-12-16T21:43:19.282199+0000","last_became_peered":"2023-12-16T21:43:19.282199+0000","last_unstale":"2023-12-16T21:43:21.279764+0000","last_undegraded":"2023-12-16T21:43:21.279764+0000","last_fullsized":"2023-12-16T21:43:21.279764+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:21:19.433920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278956+0000","last_change":"2023-12-16T21:43:20.304787+0000","last_active":"2023-12-16T21:43:21.278956+0000","last_peered":"2023-12-16T21:43:21.278956+0000","last_clean":"2023-12-16T21:43:21.278956+0000","last_became_active":"2023-12-16T21:43:20.303151+0000","last_became_peered":"2023-12-16T21:43:20.303151+0000","last_unstale":"2023-12-16T21:43:21.278956+0000","last_undegraded":"2023-12-16T21:43:21.278956+0000","last_fullsized":"2023-12-16T21:43:21.278956+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:41.036474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367807+0000","last_change":"2023-12-16T21:43:19.279774+0000","last_active":"2023-12-16T21:43:21.367807+0000","last_peered":"2023-12-16T21:43:21.367807+0000","last_clean":"2023-12-16T21:43:21.367807+0000","last_became_active":"2023-12-16T21:43:19.279328+0000","last_became_peered":"2023-12-16T21:43:19.279328+0000","last_unstale":"2023-12-16T21:43:21.367807+0000","last_undegraded":"2023-12-16T21:43:21.367807+0000","last_fullsized":"2023-12-16T21:43:21.367807+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:00:18.392895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368029+0000","last_change":"2023-12-16T21:43:20.325276+0000","last_active":"2023-12-16T21:43:21.368029+0000","last_peered":"2023-12-16T21:43:21.368029+0000","last_clean":"2023-12-16T21:43:21.368029+0000","last_became_active":"2023-12-16T21:43:20.323914+0000","last_became_peered":"2023-12-16T21:43:20.323914+0000","last_unstale":"2023-12-16T21:43:21.368029+0000","last_undegraded":"2023-12-16T21:43:21.368029+0000","last_fullsized":"2023-12-16T21:43:21.368029+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:49:43.351056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279054+0000","last_change":"2023-12-16T21:43:19.284383+0000","last_active":"2023-12-16T21:43:21.279054+0000","last_peered":"2023-12-16T21:43:21.279054+0000","last_clean":"2023-12-16T21:43:21.279054+0000","last_became_active":"2023-12-16T21:43:19.283848+0000","last_became_peered":"2023-12-16T21:43:19.283848+0000","last_unstale":"2023-12-16T21:43:21.279054+0000","last_undegraded":"2023-12-16T21:43:21.279054+0000","last_fullsized":"2023-12-16T21:43:21.279054+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:48:25.390230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371320+0000","last_change":"2023-12-16T21:43:20.321562+0000","last_active":"2023-12-16T21:43:21.371320+0000","last_peered":"2023-12-16T21:43:21.371320+0000","last_clean":"2023-12-16T21:43:21.371320+0000","last_became_active":"2023-12-16T21:43:20.321119+0000","last_became_peered":"2023-12-16T21:43:20.321119+0000","last_unstale":"2023-12-16T21:43:21.371320+0000","last_undegraded":"2023-12-16T21:43:21.371320+0000","last_fullsized":"2023-12-16T21:43:21.371320+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:15:29.187303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368984+0000","last_change":"2023-12-16T21:43:19.284609+0000","last_active":"2023-12-16T21:43:21.368984+0000","last_peered":"2023-12-16T21:43:21.368984+0000","last_clean":"2023-12-16T21:43:21.368984+0000","last_became_active":"2023-12-16T21:43:19.284250+0000","last_became_peered":"2023-12-16T21:43:19.284250+0000","last_unstale":"2023-12-16T21:43:21.368984+0000","last_undegraded":"2023-12-16T21:43:21.368984+0000","last_fullsized":"2023-12-16T21:43:21.368984+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:15.882102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369041+0000","last_change":"2023-12-16T21:43:20.321125+0000","last_active":"2023-12-16T21:43:21.369041+0000","last_peered":"2023-12-16T21:43:21.369041+0000","last_clean":"2023-12-16T21:43:21.369041+0000","last_became_active":"2023-12-16T21:43:20.320390+0000","last_became_peered":"2023-12-16T21:43:20.320390+0000","last_unstale":"2023-12-16T21:43:21.369041+0000","last_undegraded":"2023-12-16T21:43:21.369041+0000","last_fullsized":"2023-12-16T21:43:21.369041+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:21:43.837309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275431+0000","last_change":"2023-12-16T21:43:19.282877+0000","last_active":"2023-12-16T21:43:21.275431+0000","last_peered":"2023-12-16T21:43:21.275431+0000","last_clean":"2023-12-16T21:43:21.275431+0000","last_became_active":"2023-12-16T21:43:19.282447+0000","last_became_peered":"2023-12-16T21:43:19.282447+0000","last_unstale":"2023-12-16T21:43:21.275431+0000","last_undegraded":"2023-12-16T21:43:21.275431+0000","last_fullsized":"2023-12-16T21:43:21.275431+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:23:17.386717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368271+0000","last_change":"2023-12-16T21:43:20.313362+0000","last_active":"2023-12-16T21:43:21.368271+0000","last_peered":"2023-12-16T21:43:21.368271+0000","last_clean":"2023-12-16T21:43:21.368271+0000","last_became_active":"2023-12-16T21:43:20.312656+0000","last_became_peered":"2023-12-16T21:43:20.312656+0000","last_unstale":"2023-12-16T21:43:21.368271+0000","last_undegraded":"2023-12-16T21:43:21.368271+0000","last_fullsized":"2023-12-16T21:43:21.368271+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:32:57.024768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371150+0000","last_change":"2023-12-16T21:43:19.284074+0000","last_active":"2023-12-16T21:43:21.371150+0000","last_peered":"2023-12-16T21:43:21.371150+0000","last_clean":"2023-12-16T21:43:21.371150+0000","last_became_active":"2023-12-16T21:43:19.283391+0000","last_became_peered":"2023-12-16T21:43:19.283391+0000","last_unstale":"2023-12-16T21:43:21.371150+0000","last_undegraded":"2023-12-16T21:43:21.371150+0000","last_fullsized":"2023-12-16T21:43:21.371150+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:39:42.338944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370540+0000","last_change":"2023-12-16T21:43:20.328772+0000","last_active":"2023-12-16T21:43:21.370540+0000","last_peered":"2023-12-16T21:43:21.370540+0000","last_clean":"2023-12-16T21:43:21.370540+0000","last_became_active":"2023-12-16T21:43:20.328229+0000","last_became_peered":"2023-12-16T21:43:20.328229+0000","last_unstale":"2023-12-16T21:43:21.370540+0000","last_undegraded":"2023-12-16T21:43:21.370540+0000","last_fullsized":"2023-12-16T21:43:21.370540+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:40:31.341452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368501+0000","last_change":"2023-12-16T21:43:19.283068+0000","last_active":"2023-12-16T21:43:21.368501+0000","last_peered":"2023-12-16T21:43:21.368501+0000","last_clean":"2023-12-16T21:43:21.368501+0000","last_became_active":"2023-12-16T21:43:19.281758+0000","last_became_peered":"2023-12-16T21:43:19.281758+0000","last_unstale":"2023-12-16T21:43:21.368501+0000","last_undegraded":"2023-12-16T21:43:21.368501+0000","last_fullsized":"2023-12-16T21:43:21.368501+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:23:06.196965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368398+0000","last_change":"2023-12-16T21:43:20.323107+0000","last_active":"2023-12-16T21:43:21.368398+0000","last_peered":"2023-12-16T21:43:21.368398+0000","last_clean":"2023-12-16T21:43:21.368398+0000","last_became_active":"2023-12-16T21:43:20.322469+0000","last_became_peered":"2023-12-16T21:43:20.322469+0000","last_unstale":"2023-12-16T21:43:21.368398+0000","last_undegraded":"2023-12-16T21:43:21.368398+0000","last_fullsized":"2023-12-16T21:43:21.368398+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:00:19.396579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279753+0000","last_change":"2023-12-16T21:43:19.288795+0000","last_active":"2023-12-16T21:43:21.279753+0000","last_peered":"2023-12-16T21:43:21.279753+0000","last_clean":"2023-12-16T21:43:21.279753+0000","last_became_active":"2023-12-16T21:43:19.284406+0000","last_became_peered":"2023-12-16T21:43:19.284406+0000","last_unstale":"2023-12-16T21:43:21.279753+0000","last_undegraded":"2023-12-16T21:43:21.279753+0000","last_fullsized":"2023-12-16T21:43:21.279753+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:57:15.015059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.369041+0000","last_change":"2023-12-16T21:43:20.310442+0000","last_active":"2023-12-16T21:43:21.369041+0000","last_peered":"2023-12-16T21:43:21.369041+0000","last_clean":"2023-12-16T21:43:21.369041+0000","last_became_active":"2023-12-16T21:43:20.309658+0000","last_became_peered":"2023-12-16T21:43:20.309658+0000","last_unstale":"2023-12-16T21:43:21.369041+0000","last_undegraded":"2023-12-16T21:43:21.369041+0000","last_fullsized":"2023-12-16T21:43:21.369041+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:48:13.371146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368434+0000","last_change":"2023-12-16T21:43:19.275510+0000","last_active":"2023-12-16T21:43:21.368434+0000","last_peered":"2023-12-16T21:43:21.368434+0000","last_clean":"2023-12-16T21:43:21.368434+0000","last_became_active":"2023-12-16T21:43:19.275050+0000","last_became_peered":"2023-12-16T21:43:19.275050+0000","last_unstale":"2023-12-16T21:43:21.368434+0000","last_undegraded":"2023-12-16T21:43:21.368434+0000","last_fullsized":"2023-12-16T21:43:21.368434+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:40.032790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.278011+0000","last_change":"2023-12-16T21:43:20.315034+0000","last_active":"2023-12-16T21:43:21.278011+0000","last_peered":"2023-12-16T21:43:21.278011+0000","last_clean":"2023-12-16T21:43:21.278011+0000","last_became_active":"2023-12-16T21:43:20.313238+0000","last_became_peered":"2023-12-16T21:43:20.313238+0000","last_unstale":"2023-12-16T21:43:21.278011+0000","last_undegraded":"2023-12-16T21:43:21.278011+0000","last_fullsized":"2023-12-16T21:43:21.278011+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:48:13.371146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367888+0000","last_change":"2023-12-16T21:43:19.280980+0000","last_active":"2023-12-16T21:43:21.367888+0000","last_peered":"2023-12-16T21:43:21.367888+0000","last_clean":"2023-12-16T21:43:21.367888+0000","last_became_active":"2023-12-16T21:43:19.280515+0000","last_became_peered":"2023-12-16T21:43:19.280515+0000","last_unstale":"2023-12-16T21:43:21.367888+0000","last_undegraded":"2023-12-16T21:43:21.367888+0000","last_fullsized":"2023-12-16T21:43:21.367888+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:25:01.268486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370619+0000","last_change":"2023-12-16T21:43:20.300148+0000","last_active":"2023-12-16T21:43:21.370619+0000","last_peered":"2023-12-16T21:43:21.370619+0000","last_clean":"2023-12-16T21:43:21.370619+0000","last_became_active":"2023-12-16T21:43:20.299228+0000","last_became_peered":"2023-12-16T21:43:20.299228+0000","last_unstale":"2023-12-16T21:43:21.370619+0000","last_undegraded":"2023-12-16T21:43:21.370619+0000","last_fullsized":"2023-12-16T21:43:21.370619+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:41.036474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367550+0000","last_change":"2023-12-16T21:43:19.283159+0000","last_active":"2023-12-16T21:43:21.367550+0000","last_peered":"2023-12-16T21:43:21.367550+0000","last_clean":"2023-12-16T21:43:21.367550+0000","last_became_active":"2023-12-16T21:43:19.281792+0000","last_became_peered":"2023-12-16T21:43:19.281792+0000","last_unstale":"2023-12-16T21:43:21.367550+0000","last_undegraded":"2023-12-16T21:43:21.367550+0000","last_fullsized":"2023-12-16T21:43:21.367550+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:49:42.347373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279852+0000","last_change":"2023-12-16T21:43:20.323769+0000","last_active":"2023-12-16T21:43:21.279852+0000","last_peered":"2023-12-16T21:43:21.279852+0000","last_clean":"2023-12-16T21:43:21.279852+0000","last_became_active":"2023-12-16T21:43:20.323193+0000","last_became_peered":"2023-12-16T21:43:20.323193+0000","last_unstale":"2023-12-16T21:43:21.279852+0000","last_undegraded":"2023-12-16T21:43:21.279852+0000","last_fullsized":"2023-12-16T21:43:21.279852+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:39:43.342628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370088+0000","last_change":"2023-12-16T21:43:19.287872+0000","last_active":"2023-12-16T21:43:21.370088+0000","last_peered":"2023-12-16T21:43:21.370088+0000","last_clean":"2023-12-16T21:43:21.370088+0000","last_became_active":"2023-12-16T21:43:19.282035+0000","last_became_peered":"2023-12-16T21:43:19.282035+0000","last_unstale":"2023-12-16T21:43:21.370088+0000","last_undegraded":"2023-12-16T21:43:21.370088+0000","last_fullsized":"2023-12-16T21:43:21.370088+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:38:53.822126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371355+0000","last_change":"2023-12-16T21:43:20.322915+0000","last_active":"2023-12-16T21:43:21.371355+0000","last_peered":"2023-12-16T21:43:21.371355+0000","last_clean":"2023-12-16T21:43:21.371355+0000","last_became_active":"2023-12-16T21:43:20.322239+0000","last_became_peered":"2023-12-16T21:43:20.322239+0000","last_unstale":"2023-12-16T21:43:21.371355+0000","last_undegraded":"2023-12-16T21:43:21.371355+0000","last_fullsized":"2023-12-16T21:43:21.371355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:32:57.024768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279798+0000","last_change":"2023-12-16T21:43:19.289609+0000","last_active":"2023-12-16T21:43:21.279798+0000","last_peered":"2023-12-16T21:43:21.279798+0000","last_clean":"2023-12-16T21:43:21.279798+0000","last_became_active":"2023-12-16T21:43:19.289091+0000","last_became_peered":"2023-12-16T21:43:19.289091+0000","last_unstale":"2023-12-16T21:43:21.279798+0000","last_undegraded":"2023-12-16T21:43:21.279798+0000","last_fullsized":"2023-12-16T21:43:21.279798+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:41:46.897269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368677+0000","last_change":"2023-12-16T21:43:20.310115+0000","last_active":"2023-12-16T21:43:21.368677+0000","last_peered":"2023-12-16T21:43:21.368677+0000","last_clean":"2023-12-16T21:43:21.368677+0000","last_became_active":"2023-12-16T21:43:20.309289+0000","last_became_peered":"2023-12-16T21:43:20.309289+0000","last_unstale":"2023-12-16T21:43:21.368677+0000","last_undegraded":"2023-12-16T21:43:21.368677+0000","last_fullsized":"2023-12-16T21:43:21.368677+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:23:07.200649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280119+0000","last_change":"2023-12-16T21:43:19.284529+0000","last_active":"2023-12-16T21:43:21.280119+0000","last_peered":"2023-12-16T21:43:21.280119+0000","last_clean":"2023-12-16T21:43:21.280119+0000","last_became_active":"2023-12-16T21:43:19.283815+0000","last_became_peered":"2023-12-16T21:43:19.283815+0000","last_unstale":"2023-12-16T21:43:21.280119+0000","last_undegraded":"2023-12-16T21:43:21.280119+0000","last_fullsized":"2023-12-16T21:43:21.280119+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:03:02.840614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368061+0000","last_change":"2023-12-16T21:43:20.312410+0000","last_active":"2023-12-16T21:43:21.368061+0000","last_peered":"2023-12-16T21:43:21.368061+0000","last_clean":"2023-12-16T21:43:21.368061+0000","last_became_active":"2023-12-16T21:43:20.311742+0000","last_became_peered":"2023-12-16T21:43:20.311742+0000","last_unstale":"2023-12-16T21:43:21.368061+0000","last_undegraded":"2023-12-16T21:43:21.368061+0000","last_fullsized":"2023-12-16T21:43:21.368061+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:03:03.844297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280423+0000","last_change":"2023-12-16T21:43:19.284360+0000","last_active":"2023-12-16T21:43:21.280423+0000","last_peered":"2023-12-16T21:43:21.280423+0000","last_clean":"2023-12-16T21:43:21.280423+0000","last_became_active":"2023-12-16T21:43:19.283588+0000","last_became_peered":"2023-12-16T21:43:19.283588+0000","last_unstale":"2023-12-16T21:43:21.280423+0000","last_undegraded":"2023-12-16T21:43:21.280423+0000","last_fullsized":"2023-12-16T21:43:21.280423+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:58:34.582538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367537+0000","last_change":"2023-12-16T21:43:20.292017+0000","last_active":"2023-12-16T21:43:21.367537+0000","last_peered":"2023-12-16T21:43:21.367537+0000","last_clean":"2023-12-16T21:43:21.367537+0000","last_became_active":"2023-12-16T21:43:20.289647+0000","last_became_peered":"2023-12-16T21:43:20.289647+0000","last_unstale":"2023-12-16T21:43:21.367537+0000","last_undegraded":"2023-12-16T21:43:21.367537+0000","last_fullsized":"2023-12-16T21:43:21.367537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:58:35.586222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275585+0000","last_change":"2023-12-16T21:43:19.276671+0000","last_active":"2023-12-16T21:43:21.275585+0000","last_peered":"2023-12-16T21:43:21.275585+0000","last_clean":"2023-12-16T21:43:21.275585+0000","last_became_active":"2023-12-16T21:43:19.276346+0000","last_became_peered":"2023-12-16T21:43:19.276346+0000","last_unstale":"2023-12-16T21:43:21.275585+0000","last_undegraded":"2023-12-16T21:43:21.275585+0000","last_fullsized":"2023-12-16T21:43:21.275585+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:55:02.055360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367828+0000","last_change":"2023-12-16T21:43:20.302656+0000","last_active":"2023-12-16T21:43:21.367828+0000","last_peered":"2023-12-16T21:43:21.367828+0000","last_clean":"2023-12-16T21:43:21.367828+0000","last_became_active":"2023-12-16T21:43:20.301387+0000","last_became_peered":"2023-12-16T21:43:20.301387+0000","last_unstale":"2023-12-16T21:43:21.367828+0000","last_undegraded":"2023-12-16T21:43:21.367828+0000","last_fullsized":"2023-12-16T21:43:21.367828+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:41:47.900953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370403+0000","last_change":"2023-12-16T21:43:19.283687+0000","last_active":"2023-12-16T21:43:21.370403+0000","last_peered":"2023-12-16T21:43:21.370403+0000","last_clean":"2023-12-16T21:43:21.370403+0000","last_became_active":"2023-12-16T21:43:19.283020+0000","last_became_peered":"2023-12-16T21:43:19.283020+0000","last_unstale":"2023-12-16T21:43:21.370403+0000","last_undegraded":"2023-12-16T21:43:21.370403+0000","last_fullsized":"2023-12-16T21:43:21.370403+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:57:15.015059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367324+0000","last_change":"2023-12-16T21:43:20.300410+0000","last_active":"2023-12-16T21:43:21.367324+0000","last_peered":"2023-12-16T21:43:21.367324+0000","last_clean":"2023-12-16T21:43:21.367324+0000","last_became_active":"2023-12-16T21:43:20.299654+0000","last_became_peered":"2023-12-16T21:43:20.299654+0000","last_unstale":"2023-12-16T21:43:21.367324+0000","last_undegraded":"2023-12-16T21:43:21.367324+0000","last_fullsized":"2023-12-16T21:43:21.367324+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T04:12:11.288779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280064+0000","last_change":"2023-12-16T21:43:19.288868+0000","last_active":"2023-12-16T21:43:21.280064+0000","last_peered":"2023-12-16T21:43:21.280064+0000","last_clean":"2023-12-16T21:43:21.280064+0000","last_became_active":"2023-12-16T21:43:19.285277+0000","last_became_peered":"2023-12-16T21:43:19.285277+0000","last_unstale":"2023-12-16T21:43:21.280064+0000","last_undegraded":"2023-12-16T21:43:21.280064+0000","last_fullsized":"2023-12-16T21:43:21.280064+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:31:36.018521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277345+0000","last_change":"2023-12-16T21:43:20.307834+0000","last_active":"2023-12-16T21:43:21.277345+0000","last_peered":"2023-12-16T21:43:21.277345+0000","last_clean":"2023-12-16T21:43:21.277345+0000","last_became_active":"2023-12-16T21:43:20.307083+0000","last_became_peered":"2023-12-16T21:43:20.307083+0000","last_unstale":"2023-12-16T21:43:21.277345+0000","last_undegraded":"2023-12-16T21:43:21.277345+0000","last_fullsized":"2023-12-16T21:43:21.277345+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:31:16.885786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370375+0000","last_change":"2023-12-16T21:43:19.284519+0000","last_active":"2023-12-16T21:43:21.370375+0000","last_peered":"2023-12-16T21:43:21.370375+0000","last_clean":"2023-12-16T21:43:21.370375+0000","last_became_active":"2023-12-16T21:43:19.283895+0000","last_became_peered":"2023-12-16T21:43:19.283895+0000","last_unstale":"2023-12-16T21:43:21.370375+0000","last_undegraded":"2023-12-16T21:43:21.370375+0000","last_fullsized":"2023-12-16T21:43:21.370375+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T09:41:40.032790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368246+0000","last_change":"2023-12-16T21:43:20.321897+0000","last_active":"2023-12-16T21:43:21.368246+0000","last_peered":"2023-12-16T21:43:21.368246+0000","last_clean":"2023-12-16T21:43:21.368246+0000","last_became_active":"2023-12-16T21:43:20.321591+0000","last_became_peered":"2023-12-16T21:43:20.321591+0000","last_unstale":"2023-12-16T21:43:21.368246+0000","last_undegraded":"2023-12-16T21:43:21.368246+0000","last_fullsized":"2023-12-16T21:43:21.368246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:31:37.022205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370102+0000","last_change":"2023-12-16T21:43:19.279935+0000","last_active":"2023-12-16T21:43:21.370102+0000","last_peered":"2023-12-16T21:43:21.370102+0000","last_clean":"2023-12-16T21:43:21.370102+0000","last_became_active":"2023-12-16T21:43:19.279304+0000","last_became_peered":"2023-12-16T21:43:19.279304+0000","last_unstale":"2023-12-16T21:43:21.370102+0000","last_undegraded":"2023-12-16T21:43:21.370102+0000","last_fullsized":"2023-12-16T21:43:21.370102+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T07:39:42.338944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.371666+0000","last_change":"2023-12-16T21:43:20.321762+0000","last_active":"2023-12-16T21:43:21.371666+0000","last_peered":"2023-12-16T21:43:21.371666+0000","last_clean":"2023-12-16T21:43:21.371666+0000","last_became_active":"2023-12-16T21:43:20.321157+0000","last_became_peered":"2023-12-16T21:43:20.321157+0000","last_unstale":"2023-12-16T21:43:21.371666+0000","last_undegraded":"2023-12-16T21:43:21.371666+0000","last_fullsized":"2023-12-16T21:43:21.371666+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T05:48:13.371146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275554+0000","last_change":"2023-12-16T21:43:19.271444+0000","last_active":"2023-12-16T21:43:21.275554+0000","last_peered":"2023-12-16T21:43:21.275554+0000","last_clean":"2023-12-16T21:43:21.275554+0000","last_became_active":"2023-12-16T21:43:19.271083+0000","last_became_peered":"2023-12-16T21:43:19.271083+0000","last_unstale":"2023-12-16T21:43:21.275554+0000","last_undegraded":"2023-12-16T21:43:21.275554+0000","last_fullsized":"2023-12-16T21:43:21.275554+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:21:19.433920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275250+0000","last_change":"2023-12-16T21:43:20.322719+0000","last_active":"2023-12-16T21:43:21.275250+0000","last_peered":"2023-12-16T21:43:21.275250+0000","last_clean":"2023-12-16T21:43:21.275250+0000","last_became_active":"2023-12-16T21:43:20.322022+0000","last_became_peered":"2023-12-16T21:43:20.322022+0000","last_unstale":"2023-12-16T21:43:21.275250+0000","last_undegraded":"2023-12-16T21:43:21.275250+0000","last_fullsized":"2023-12-16T21:43:21.275250+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:43.887172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368801+0000","last_change":"2023-12-16T21:43:19.274044+0000","last_active":"2023-12-16T21:43:21.368801+0000","last_peered":"2023-12-16T21:43:21.368801+0000","last_clean":"2023-12-16T21:43:21.368801+0000","last_became_active":"2023-12-16T21:43:19.269096+0000","last_became_peered":"2023-12-16T21:43:19.269096+0000","last_unstale":"2023-12-16T21:43:21.368801+0000","last_undegraded":"2023-12-16T21:43:21.368801+0000","last_fullsized":"2023-12-16T21:43:21.368801+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:36:42.883488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370456+0000","last_change":"2023-12-16T21:43:20.305227+0000","last_active":"2023-12-16T21:43:21.370456+0000","last_peered":"2023-12-16T21:43:21.370456+0000","last_clean":"2023-12-16T21:43:21.370456+0000","last_became_active":"2023-12-16T21:43:20.304565+0000","last_became_peered":"2023-12-16T21:43:20.304565+0000","last_unstale":"2023-12-16T21:43:21.370456+0000","last_undegraded":"2023-12-16T21:43:21.370456+0000","last_fullsized":"2023-12-16T21:43:21.370456+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T08:17:02.836979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275683+0000","last_change":"2023-12-16T21:43:19.274011+0000","last_active":"2023-12-16T21:43:21.275683+0000","last_peered":"2023-12-16T21:43:21.275683+0000","last_clean":"2023-12-16T21:43:21.275683+0000","last_became_active":"2023-12-16T21:43:19.273273+0000","last_became_peered":"2023-12-16T21:43:19.273273+0000","last_unstale":"2023-12-16T21:43:21.275683+0000","last_undegraded":"2023-12-16T21:43:21.275683+0000","last_fullsized":"2023-12-16T21:43:21.275683+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:57:15.015059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.275332+0000","last_change":"2023-12-16T21:43:20.306745+0000","last_active":"2023-12-16T21:43:21.275332+0000","last_peered":"2023-12-16T21:43:21.275332+0000","last_clean":"2023-12-16T21:43:21.275332+0000","last_became_active":"2023-12-16T21:43:20.305961+0000","last_became_peered":"2023-12-16T21:43:20.305961+0000","last_unstale":"2023-12-16T21:43:21.275332+0000","last_undegraded":"2023-12-16T21:43:21.275332+0000","last_fullsized":"2023-12-16T21:43:21.275332+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:38:54.825810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.368516+0000","last_change":"2023-12-16T21:43:19.279580+0000","last_active":"2023-12-16T21:43:21.368516+0000","last_peered":"2023-12-16T21:43:21.368516+0000","last_clean":"2023-12-16T21:43:21.368516+0000","last_became_active":"2023-12-16T21:43:19.279134+0000","last_became_peered":"2023-12-16T21:43:19.279134+0000","last_unstale":"2023-12-16T21:43:21.368516+0000","last_undegraded":"2023-12-16T21:43:21.368516+0000","last_fullsized":"2023-12-16T21:43:21.368516+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:38.002112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367417+0000","last_change":"2023-12-16T21:43:20.318172+0000","last_active":"2023-12-16T21:43:21.367417+0000","last_peered":"2023-12-16T21:43:21.367417+0000","last_clean":"2023-12-16T21:43:21.367417+0000","last_became_active":"2023-12-16T21:43:20.317616+0000","last_became_peered":"2023-12-16T21:43:20.317616+0000","last_unstale":"2023-12-16T21:43:21.367417+0000","last_undegraded":"2023-12-16T21:43:21.367417+0000","last_fullsized":"2023-12-16T21:43:21.367417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:46:40.185321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.280293+0000","last_change":"2023-12-16T21:43:19.285249+0000","last_active":"2023-12-16T21:43:21.280293+0000","last_peered":"2023-12-16T21:43:21.280293+0000","last_clean":"2023-12-16T21:43:21.280293+0000","last_became_active":"2023-12-16T21:43:19.284732+0000","last_became_peered":"2023-12-16T21:43:19.284732+0000","last_unstale":"2023-12-16T21:43:21.280293+0000","last_undegraded":"2023-12-16T21:43:21.280293+0000","last_fullsized":"2023-12-16T21:43:21.280293+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T02:07:12.587135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.279431+0000","last_change":"2023-12-16T21:43:20.311137+0000","last_active":"2023-12-16T21:43:21.279431+0000","last_peered":"2023-12-16T21:43:21.279431+0000","last_clean":"2023-12-16T21:43:21.279431+0000","last_became_active":"2023-12-16T21:43:20.310423+0000","last_became_peered":"2023-12-16T21:43:20.310423+0000","last_unstale":"2023-12-16T21:43:21.279431+0000","last_undegraded":"2023-12-16T21:43:21.279431+0000","last_fullsized":"2023-12-16T21:43:21.279431+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T22:58:39.005796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.367667+0000","last_change":"2023-12-16T21:43:19.287996+0000","last_active":"2023-12-16T21:43:21.367667+0000","last_peered":"2023-12-16T21:43:21.367667+0000","last_clean":"2023-12-16T21:43:21.367667+0000","last_became_active":"2023-12-16T21:43:19.286719+0000","last_became_peered":"2023-12-16T21:43:19.286719+0000","last_unstale":"2023-12-16T21:43:21.367667+0000","last_undegraded":"2023-12-16T21:43:21.367667+0000","last_fullsized":"2023-12-16T21:43:21.367667+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T03:32:36.251240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.277701+0000","last_change":"2023-12-16T21:43:20.298745+0000","last_active":"2023-12-16T21:43:21.277701+0000","last_peered":"2023-12-16T21:43:21.277701+0000","last_clean":"2023-12-16T21:43:21.277701+0000","last_became_active":"2023-12-16T21:43:20.298258+0000","last_became_peered":"2023-12-16T21:43:20.298258+0000","last_unstale":"2023-12-16T21:43:21.277701+0000","last_undegraded":"2023-12-16T21:43:21.277701+0000","last_fullsized":"2023-12-16T21:43:21.277701+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:19.239844+0000","last_clean_scrub_stamp":"2023-12-16T21:43:19.239844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-17T23:32:57.024768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-12-16T21:43:21.370116+0000","last_change":"2023-12-16T21:43:19.293689+0000","last_active":"2023-12-16T21:43:21.370116+0000","last_peered":"2023-12-16T21:43:21.370116+0000","last_clean":"2023-12-16T21:43:21.370116+0000","last_became_active":"2023-12-16T21:43:19.293025+0000","last_became_peered":"2023-12-16T21:43:19.293025+0000","last_unstale":"2023-12-16T21:43:21.370116+0000","last_undegraded":"2023-12-16T21:43:21.370116+0000","last_fullsized":"2023-12-16T21:43:21.370116+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:43:18.236160+0000","last_clean_scrub_stamp":"2023-12-16T21:43:18.236160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T06:27:20.836500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-16T21:43:20.286524+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:20.286524+0000","last_peered":"2023-12-16T21:43:20.286524+0000","last_clean":"2023-12-16T21:43:20.286524+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:20.286524+0000","last_undegraded":"2023-12-16T21:43:20.286524+0000","last_fullsized":"2023-12-16T21:43:20.286524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":719117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.038}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":129275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.222}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":129331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":128763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.036}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0640000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":352256,"data_stored":130293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":339968,"data_stored":128729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":352256,"data_stored":130293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0529999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.224}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32100,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339740,"statfs":{"total":96636764160,"available":96603893760,"internally_reserved":0,"allocated":929792,"data_stored":719027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.516}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:28.171 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-16T21:43:28.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-16T21:43:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.331+0000 7f8457c77640 1 Processor -- start 2023-12-16T21:43:28.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.331+0000 7f8457c77640 1 -- start start 2023-12-16T21:43:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8457c77640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500af300 0x7f8450144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8457c77640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84500b4350 0x7f8450144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8457c77640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84500b1730 con 0x7f84500b1eb0 2023-12-16T21:43:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8457c77640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845006cd70 con 0x7f84500b4350 2023-12-16T21:43:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8457c77640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84500b0ad0 con 0x7f84500af300 2023-12-16T21:43:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8455212640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500af300 0x7f8450144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.339 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8456214640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84500b4350 0x7f8450144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8456214640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f84500b4350 0x7f8450144ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:60056/0 (socket says 172.21.15.46:60056) 2023-12-16T21:43:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8456214640 1 -- 172.21.15.46:0/2496482745 learned_addr learned my addr 172.21.15.46:0/2496482745 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8456214640 1 -- 172.21.15.46:0/2496482745 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500af300 msgr2=0x7f8450144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8456214640 1 --2- 172.21.15.46:0/2496482745 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500af300 0x7f8450144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8456214640 1 -- 172.21.15.46:0/2496482745 >> v1:172.21.15.161:6789/0 conn(0x7f84500b1eb0 legacy=0x7f84500aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.340 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.335+0000 7f8456214640 1 -- 172.21.15.46:0/2496482745 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845006c7f0 con 0x7f84500b4350 2023-12-16T21:43:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8456214640 1 --2- 172.21.15.46:0/2496482745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f8450144ff0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8448009870 tx=0x7f844801c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=caeaac68be13f8a server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8454a11640 1 -- 172.21.15.46:0/2496482745 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84480ab620 con 0x7f84500b4350 2023-12-16T21:43:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8454a11640 1 -- 172.21.15.46:0/2496482745 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84480ab780 con 0x7f84500b4350 2023-12-16T21:43:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8454a11640 1 -- 172.21.15.46:0/2496482745 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84480abb10 con 0x7f84500b4350 2023-12-16T21:43:28.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- 172.21.15.46:0/2496482745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 msgr2=0x7f8450144ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 --2- 172.21.15.46:0/2496482745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f8450144ff0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8448009870 tx=0x7f844801c420 comp rx=0 tx=0).stop 2023-12-16T21:43:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- 172.21.15.46:0/2496482745 shutdown_connections 2023-12-16T21:43:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 --2- 172.21.15.46:0/2496482745 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f8450144ff0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 --2- 172.21.15.46:0/2496482745 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500af300 0x7f8450144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- 172.21.15.46:0/2496482745 >> 172.21.15.46:0/2496482745 conn(0x7f84500b4900 msgr2=0x7f84500b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- 172.21.15.46:0/2496482745 shutdown_connections 2023-12-16T21:43:28.343 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- 172.21.15.46:0/2496482745 wait complete. 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 Processor -- start 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- start start 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84500af300 0x7f8450139530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500b1eb0 0x7f845006fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f84501df960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845006e840 con 0x7f84500af300 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845013ffd0 con 0x7f84500b4350 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8457c77640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84501402b0 con 0x7f84500b1eb0 2023-12-16T21:43:28.346 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84500af300 0x7f8450139530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84500af300 0x7f8450139530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53918/0 (socket says 172.21.15.46:53918) 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 -- 172.21.15.46:0/765436294 learned_addr learned my addr 172.21.15.46:0/765436294 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8456214640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f84501df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455212640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500b1eb0 0x7f845006fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 -- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500b1eb0 msgr2=0x7f845006fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500b1eb0 0x7f845006fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 -- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 msgr2=0x7f84501df960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f84501df960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 -- 172.21.15.46:0/765436294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84480094d0 con 0x7f84500af300 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8456214640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f84501df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f8455a13640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84500af300 0x7f8450139530 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f844000c7f0 tx=0x7f844000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f844e7fc640 1 -- 172.21.15.46:0/765436294 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8440003b80 con 0x7f84500af300 2023-12-16T21:43:28.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f844e7fc640 1 -- 172.21.15.46:0/765436294 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8440003ce0 con 0x7f84500af300 2023-12-16T21:43:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.339+0000 7f844e7fc640 1 -- 172.21.15.46:0/765436294 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844000aa90 con 0x7f84500af300 2023-12-16T21:43:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84501445f0 con 0x7f84500af300 2023-12-16T21:43:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845006ed70 con 0x7f84500af300 2023-12-16T21:43:28.348 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8450068960 con 0x7f84500af300 2023-12-16T21:43:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f844e7fc640 1 -- 172.21.15.46:0/765436294 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f844000abf0 con 0x7f84500af300 2023-12-16T21:43:28.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f844e7fc640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f843c066470 0x7f843c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f844e7fc640 1 -- 172.21.15.46:0/765436294 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8440083cd0 con 0x7f84500af300 2023-12-16T21:43:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f8455212640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f843c066470 0x7f843c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.343+0000 7f8455212640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f843c066470 0x7f843c068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8450134270 tx=0x7f8444008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:28.352 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.347+0000 7f844e7fc640 1 -- 172.21.15.46:0/765436294 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f84400511e0 con 0x7f84500af300 2023-12-16T21:43:28.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.559+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f84500703a0 con 0x7f84500af300 2023-12-16T21:43:28.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.563+0000 7f844e7fc640 1 -- 172.21.15.46:0/765436294 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8440054800 con 0x7f84500af300 2023-12-16T21:43:28.568 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2023-12-16T21:43:28.568 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCSGX5lKFNZNRAAtR1lv++0meNbUxdnBSKAGg== 2023-12-16T21:43:28.568 INFO:teuthology.orchestra.run.smithi046.stdout: caps mds = "allow" 2023-12-16T21:43:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: caps mon = "allow r" 2023-12-16T21:43:28.569 INFO:teuthology.orchestra.run.smithi046.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-12-16T21:43:28.569 INFO:teuthology.orchestra.run.smithi046.stderr:updated caps for client.0 2023-12-16T21:43:28.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f843c066470 msgr2=0x7f843c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f843c066470 0x7f843c068930 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8450134270 tx=0x7f8444008040 comp rx=0 tx=0).stop 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84500af300 msgr2=0x7f8450139530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84500af300 0x7f8450139530 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f844000c7f0 tx=0x7f844000ccb0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 shutdown_connections 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f843c066470 0x7f843c068930 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f84500b4350 0x7f84501df960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f84500b1eb0 0x7f845006fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 --2- 172.21.15.46:0/765436294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84500af300 0x7f8450139530 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 >> 172.21.15.46:0/765436294 conn(0x7f84500b4900 msgr2=0x7f84501423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 shutdown_connections 2023-12-16T21:43:28.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.567+0000 7f8457c77640 1 -- 172.21.15.46:0/765436294 wait complete. 2023-12-16T21:43:28.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284bef6640 1 Processor -- start 2023-12-16T21:43:28.705 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284bef6640 1 -- start start 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284bef6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28440bc800 0x7f28440b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284bef6640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f284413ff50 0x7f28441403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284bef6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28440bcbe0 con 0x7f2844149c80 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284bef6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28440b38f0 con 0x7f28440bc800 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284bef6640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28440b3bd0 con 0x7f284413ff50 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849c92640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f28440bc800 0x7f28440b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849491640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f2844149c80 0x7f284414a110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56062/0 (socket says 172.21.15.46:56062) 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849c92640 1 --2- 172.21.15.46:0/1263617601 >> v2:172.21.15.46:3300/0 conn(0x7f28440bc800 0x7f28440b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50050/0 (socket says 172.21.15.46:50050) 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849491640 1 -- 172.21.15.46:0/1263617601 learned_addr learned my addr 172.21.15.46:0/1263617601 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f284a493640 1 --2- 172.21.15.46:0/1263617601 >> v2:172.21.15.161:3300/0 conn(0x7f284413ff50 0x7f28441403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849c92640 1 -- 172.21.15.46:0/1263617601 >> v2:172.21.15.161:3300/0 conn(0x7f284413ff50 msgr2=0x7f28441403c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849c92640 1 --2- 172.21.15.46:0/1263617601 >> v2:172.21.15.161:3300/0 conn(0x7f284413ff50 0x7f28441403c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.707 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849c92640 1 -- 172.21.15.46:0/1263617601 >> v1:172.21.15.46:6789/0 conn(0x7f2844149c80 legacy=0x7f284414a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2849c92640 1 -- 172.21.15.46:0/1263617601 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28440b3eb0 con 0x7f28440bc800 2023-12-16T21:43:28.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.703+0000 7f2848c90640 1 -- 172.21.15.46:0/1263617601 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373346711 0 0) 0x7f28440bcbe0 con 0x7f2844149c80 2023-12-16T21:43:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849c92640 1 --2- 172.21.15.46:0/1263617601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28440bc800 0x7f28440b4bb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2844146590 tx=0x7f284001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de30e8eb4177dc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2848c90640 1 -- 172.21.15.46:0/1263617601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400ab600 con 0x7f28440bc800 2023-12-16T21:43:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2848c90640 1 -- 172.21.15.46:0/1263617601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28400a7050 con 0x7f28440bc800 2023-12-16T21:43:28.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2848c90640 1 -- 172.21.15.46:0/1263617601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28400ab980 con 0x7f28440bc800 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- 172.21.15.46:0/1263617601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28440bc800 msgr2=0x7f28440b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 --2- 172.21.15.46:0/1263617601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28440bc800 0x7f28440b4bb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2844146590 tx=0x7f284001c460 comp rx=0 tx=0).stop 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- 172.21.15.46:0/1263617601 shutdown_connections 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 --2- 172.21.15.46:0/1263617601 >> v2:172.21.15.161:3300/0 conn(0x7f284413ff50 0x7f28441403c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 --2- 172.21.15.46:0/1263617601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f28440bc800 0x7f28440b4bb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- 172.21.15.46:0/1263617601 >> 172.21.15.46:0/1263617601 conn(0x7f28440722b0 msgr2=0x7f28440bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- 172.21.15.46:0/1263617601 shutdown_connections 2023-12-16T21:43:28.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- 172.21.15.46:0/1263617601 wait complete. 2023-12-16T21:43:28.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 Processor -- start 2023-12-16T21:43:28.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- start start 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28440bc800 0x7f284413a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284413ff50 0x7f28441384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2844149c80 0x7f28441e25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284413b850 con 0x7f284413ff50 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28441e1f30 con 0x7f2844149c80 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284413b1c0 con 0x7f28440bc800 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284413ff50 0x7f28441384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284413ff50 0x7f28441384e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50060/0 (socket says 172.21.15.46:50060) 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 -- 172.21.15.46:0/1529117351 learned_addr learned my addr 172.21.15.46:0/1529117351 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 -- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28440bc800 msgr2=0x7f284413a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28440bc800 0x7f284413a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 -- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2844149c80 msgr2=0x7f28441e25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2844149c80 0x7f28441e25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 -- 172.21.15.46:0/1529117351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2840009420 con 0x7f284413ff50 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f2849491640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284413ff50 0x7f28441384e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f283400c480 tx=0x7f283400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f283a7fc640 1 -- 172.21.15.46:0/1529117351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28340074e0 con 0x7f284413ff50 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f283a7fc640 1 -- 172.21.15.46:0/1529117351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2834007660 con 0x7f284413ff50 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284413d890 con 0x7f284413ff50 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.707+0000 7f283a7fc640 1 -- 172.21.15.46:0/1529117351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834004a30 con 0x7f284413ff50 2023-12-16T21:43:28.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.711+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2844139620 con 0x7f284413ff50 2023-12-16T21:43:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.711+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2844067de0 con 0x7f284413ff50 2023-12-16T21:43:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.711+0000 7f283a7fc640 1 -- 172.21.15.46:0/1529117351 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2834004b90 con 0x7f284413ff50 2023-12-16T21:43:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.711+0000 7f283a7fc640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2828066470 0x7f2828068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.711+0000 7f283a7fc640 1 -- 172.21.15.46:0/1529117351 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2834083370 con 0x7f284413ff50 2023-12-16T21:43:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.711+0000 7f2849c92640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2828066470 0x7f2828068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:28.715 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.711+0000 7f2849c92640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2828066470 0x7f2828068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f28400ad000 tx=0x7f2840008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:28.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.715+0000 7f283a7fc640 1 -- 172.21.15.46:0/1529117351 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2834054310 con 0x7f284413ff50 2023-12-16T21:43:28.910 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.907+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f284413b330 con 0x7f284413ff50 2023-12-16T21:43:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.907+0000 7f283a7fc640 1 -- 172.21.15.46:0/1529117351 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3359 (secure 0 0 0) 0x7f2834053ea0 con 0x7f284413ff50 2023-12-16T21:43:29.073 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:29.073 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3332084711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3332084711},{"type":"v1","addr":"172.21.15.161:6835","nonce":3332084711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:43:19.618214+0000","modified":"2023-12-16T21:43:21.693210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/3724256427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3724256427},{"type":"v1","addr":"172.21.15.46:6833","nonce":3724256427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-16T21:43:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2828066470 msgr2=0x7f2828068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2828066470 0x7f2828068930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f28400ad000 tx=0x7f2840008040 comp rx=0 tx=0).stop 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284413ff50 msgr2=0x7f28441384e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284413ff50 0x7f28441384e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f283400c480 tx=0x7f283400c940 comp rx=0 tx=0).stop 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 shutdown_connections 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2828066470 0x7f2828068930 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2844149c80 0x7f28441e25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f284413ff50 0x7f28441384e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 --2- 172.21.15.46:0/1529117351 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f28440bc800 0x7f284413a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 >> 172.21.15.46:0/1529117351 conn(0x7f28440722b0 msgr2=0x7f2844144590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 shutdown_connections 2023-12-16T21:43:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:28.911+0000 7f284bef6640 1 -- 172.21.15.46:0/1529117351 wait complete. 2023-12-16T21:43:29.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-16T21:43:19.618214+0000', 'modified': '2023-12-16T21:43:21.693210+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4863': {'gid': 4863, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.46:6833/3724256427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.46:6832', 'nonce': 3724256427}, {'type': 'v1', 'addr': '172.21.15.46:6833', 'nonce': 3724256427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-16T21:43:29.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-16T21:43:29.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-16T21:43:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 Processor -- start 2023-12-16T21:43:29.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 -- start start 2023-12-16T21:43:29.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f07880bc8f0 0x7f07880b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0788146720 0x7f0788146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f0788140660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07880b5220 con 0x7f0788146720 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07880bccd0 con 0x7f07880bc8f0 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078ca05640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07880b3c00 con 0x7f07881401d0 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0788146720 0x7f0788146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0788146720 0x7f0788146b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50070/0 (socket says 172.21.15.46:50070) 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 -- 172.21.15.46:0/3782260722 learned_addr learned my addr 172.21.15.46:0/3782260722 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0786d9d640 1 --2- 172.21.15.46:0/3782260722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f0788140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078659c640 1 --2- 172.21.15.46:0/3782260722 >> v2:172.21.15.161:3300/0 conn(0x7f07880bc8f0 0x7f07880b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 -- 172.21.15.46:0/3782260722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 msgr2=0x7f0788140660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.408 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 --2- 172.21.15.46:0/3782260722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f0788140660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 -- 172.21.15.46:0/3782260722 >> v2:172.21.15.161:3300/0 conn(0x7f07880bc8f0 msgr2=0x7f07880b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 --2- 172.21.15.46:0/3782260722 >> v2:172.21.15.161:3300/0 conn(0x7f07880bc8f0 0x7f07880b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f0785d9b640 1 -- 172.21.15.46:0/3782260722 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07880b3ee0 con 0x7f0788146720 2023-12-16T21:43:29.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.403+0000 7f078659c640 1 --2- 172.21.15.46:0/3782260722 >> v2:172.21.15.161:3300/0 conn(0x7f07880bc8f0 0x7f07880b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f0785d9b640 1 --2- 172.21.15.46:0/3782260722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0788146720 0x7f0788146b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f07880b7b10 tx=0x7f077801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738728448eae700c server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078559a640 1 -- 172.21.15.46:0/3782260722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07780ac410 con 0x7f0788146720 2023-12-16T21:43:29.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078559a640 1 -- 172.21.15.46:0/3782260722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07780ac570 con 0x7f0788146720 2023-12-16T21:43:29.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078559a640 1 -- 172.21.15.46:0/3782260722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07780ac900 con 0x7f0788146720 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 -- 172.21.15.46:0/3782260722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0788146720 msgr2=0x7f0788146b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 --2- 172.21.15.46:0/3782260722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0788146720 0x7f0788146b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f07880b7b10 tx=0x7f077801c500 comp rx=0 tx=0).stop 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 -- 172.21.15.46:0/3782260722 shutdown_connections 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 --2- 172.21.15.46:0/3782260722 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f0788140660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 --2- 172.21.15.46:0/3782260722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0788146720 0x7f0788146b80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 --2- 172.21.15.46:0/3782260722 >> v2:172.21.15.161:3300/0 conn(0x7f07880bc8f0 0x7f07880b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 -- 172.21.15.46:0/3782260722 >> 172.21.15.46:0/3782260722 conn(0x7f07880722b0 msgr2=0x7f07880bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 -- 172.21.15.46:0/3782260722 shutdown_connections 2023-12-16T21:43:29.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 -- 172.21.15.46:0/3782260722 wait complete. 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 Processor -- start 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.407+0000 7f078ca05640 1 -- start start 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880bc8f0 0x7f0788149830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f07881e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0788146720 0x7f07881e36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788149d70 con 0x7f07880bc8f0 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078814a480 con 0x7f0788146720 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078813a070 con 0x7f07881401d0 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880bc8f0 0x7f0788149830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880bc8f0 0x7f0788149830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50076/0 (socket says 172.21.15.46:50076) 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 -- 172.21.15.46:0/650957939 learned_addr learned my addr 172.21.15.46:0/650957939 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0785d9b640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f07881e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0786d9d640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0788146720 0x7f07881e36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 -- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 msgr2=0x7f07881e3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f07881e3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 -- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0788146720 msgr2=0x7f07881e36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0788146720 0x7f07881e36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 -- 172.21.15.46:0/650957939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07780094d0 con 0x7f07880bc8f0 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0785d9b640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f07881e3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078659c640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880bc8f0 0x7f0788149830 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f078000cfc0 tx=0x7f078000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0776ffd640 1 -- 172.21.15.46:0/650957939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000d840 con 0x7f07880bc8f0 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0776ffd640 1 -- 172.21.15.46:0/650957939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078000d9a0 con 0x7f07880bc8f0 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078813a590 con 0x7f07880bc8f0 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078813a360 con 0x7f07880bc8f0 2023-12-16T21:43:29.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0776ffd640 1 -- 172.21.15.46:0/650957939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000dd30 con 0x7f07880bc8f0 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0776ffd640 1 -- 172.21.15.46:0/650957939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f0780015440 con 0x7f07880bc8f0 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0776ffd640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f07680664a0 0x7f0768068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0776ffd640 1 -- 172.21.15.46:0/650957939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0780084760 con 0x7f07880bc8f0 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.411+0000 7f0785d9b640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f07680664a0 0x7f0768068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 Processor -- start 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- start start 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f07680664a0 0x7f0768068960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07780ae000 tx=0x7f07780a6040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d21041c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2104700 0x55f5d2109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2109790 0x55f5d2109b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1fa0f00 con 0x55f5d2104700 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1f30dd0 con 0x55f5d2109790 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1f2e300 con 0x55f5d2103de0 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2104700 0x55f5d2109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2104700 0x55f5d2109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50092/0 (socket says 172.21.15.46:50092) 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 -- 172.21.15.46:0/3568993186 learned_addr learned my addr 172.21.15.46:0/3568993186 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2109790 0x55f5d2109b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078659c640 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d21041c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 -- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 msgr2=0x55f5d21041c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d21041c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 -- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2109790 msgr2=0x55f5d2109b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2109790 0x55f5d2109b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 -- 172.21.15.46:0/3568993186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d1c21890 con 0x55f5d2104700 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078659c640 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d21041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2104700 0x55f5d2109250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0778000c10 tx=0x7f0778004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d9ae113300b7b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f076ffff640 1 -- 172.21.15.46:0/3568993186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07780bd070 con 0x55f5d2104700 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f076ffff640 1 -- 172.21.15.46:0/3568993186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07780c1430 con 0x55f5d2104700 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f076ffff640 1 -- 172.21.15.46:0/3568993186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07780c15a0 con 0x55f5d2104700 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- 172.21.15.46:0/3568993186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2104700 msgr2=0x55f5d2109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2104700 0x55f5d2109250 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0778000c10 tx=0x7f0778004740 comp rx=0 tx=0).stop 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- 172.21.15.46:0/3568993186 shutdown_connections 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2109790 0x55f5d2109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2104700 0x55f5d2109250 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- 172.21.15.46:0/3568993186 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d21041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- 172.21.15.46:0/3568993186 >> 172.21.15.46:0/3568993186 conn(0x55f5d2105000 msgr2=0x55f5d21073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- 172.21.15.46:0/3568993186 shutdown_connections 2023-12-16T21:43:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- 172.21.15.46:0/3568993186 wait complete. 2023-12-16T21:43:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- start start 2023-12-16T21:43:29.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 Processor -- start 2023-12-16T21:43:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d20de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2104700 0x55f5d210f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2109790 0x55f5d210fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1f29bf0 con 0x55f5d2109790 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1ee1c60 con 0x55f5d2103de0 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1ee7b00 con 0x55f5d2104700 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078fb36000 10 client.0 _ll_register_callbacks cb 0x55f5d20c4710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2109790 0x55f5d210fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2109790 0x55f5d210fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50096/0 (socket says 172.21.15.46:50096) 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 -- 172.21.15.46:0/4193159793 learned_addr learned my addr 172.21.15.46:0/4193159793 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f078659c640 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d20de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0785d9b640 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2104700 0x55f5d210f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 -- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2104700 msgr2=0x55f5d210f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2104700 0x55f5d210f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 -- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 msgr2=0x55f5d20de4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d20de4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.415+0000 7f0786d9d640 1 -- 172.21.15.46:0/4193159793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07880b3ee0 con 0x55f5d2109790 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f0786d9d640 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2109790 0x55f5d210fa90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f077c00ddd0 tx=0x7f077c00f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75ea97112152aa8 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 1 -- 172.21.15.46:0/4193159793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c014680 con 0x55f5d2109790 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 1 -- 172.21.15.46:0/4193159793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f077c0147e0 con 0x55f5d2109790 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5d1ee7490 con 0x55f5d2109790 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 1 -- 172.21.15.46:0/4193159793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f077c00fed0 con 0x55f5d2109790 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 10 client.5049 fetch_fsmap learned FSMap version 13 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55f5d1eea200 con 0x55f5d2109790 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 1 -- 172.21.15.46:0/4193159793 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5d1ee7490 con 0x55f5d2109790 2023-12-16T21:43:29.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 1 -- 172.21.15.46:0/4193159793 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 13) v1 ==== 2144+0+0 (secure 0 0 0) 0x7f077c01dd00 con 0x55f5d2109790 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 10 client.5049 fetch_fsmap finished waiting for FSMap version 13 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 10 client.5049 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 20 client.5049 populate_metadata read hostname 'smithi046' 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d20de9f0 0x55f5d21104a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 4 client.5049 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 --> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5d1f28540 con 0x55f5d20de9f0 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078659c640 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d20de9f0 0x55f5d21104a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078659c640 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d20de9f0 0x55f5d21104a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9715e58160c80274 server_cookie=db3521508d6b441c in_seq=0 out_seq=0 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 10 client.5049 ms_handle_connect on v2:172.21.15.46:6832/3724256427 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 1 -- 172.21.15.46:0/4193159793 <== mds.0 v2:172.21.15.46:6832/3724256427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f5d1f28540 con 0x55f5d20de9f0 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 10 client.5049 handle_command_reply: tid=0 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 client.5049 shutdown 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2109790 msgr2=0x55f5d210fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2109790 0x55f5d210fa90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f077c00ddd0 tx=0x7f077c00f5b0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 shutdown_connections 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d20de9f0 0x55f5d21104a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2109790 0x55f5d210fa90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2104700 0x55f5d210f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 --2- 172.21.15.46:0/4193159793 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d20de4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 >> 172.21.15.46:0/4193159793 conn(0x55f5d2105000 msgr2=0x55f5d20df060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f076d7fa640 0 client.5049 ms_handle_reset on v2:172.21.15.46:6832/3724256427 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 shutdown_connections 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 1 -- 172.21.15.46:0/4193159793 wait complete. 2023-12-16T21:43:29.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.419+0000 7f078fb36000 20 client.5049 trim_cache size 0 max 16384 2023-12-16T21:43:29.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 Processor -- start 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 -- start start 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d210a580 0x55f5d210f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2105000 0x55f5d20dd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d20dd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1be2940 con 0x55f5d2105000 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1f23ef0 con 0x55f5d210a580 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1c221b0 con 0x55f5d2103de0 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2105000 0x55f5d20dd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2105000 0x55f5d20dd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50112/0 (socket says 172.21.15.46:50112) 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 -- 172.21.15.46:0/773483370 learned_addr learned my addr 172.21.15.46:0/773483370 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f078659c640 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d210a580 0x55f5d210f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0786d9d640 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d20dd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 -- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 msgr2=0x55f5d20dd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d20dd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 -- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d210a580 msgr2=0x55f5d210f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d210a580 0x55f5d210f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 -- 172.21.15.46:0/773483370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5d1ed0500 con 0x55f5d2105000 2023-12-16T21:43:29.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.435+0000 7f0785d9b640 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2105000 0x55f5d20dd040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f07780c4040 tx=0x7f07780b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b6d5b856388f5a server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f0774ff9640 1 -- 172.21.15.46:0/773483370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778006a70 con 0x55f5d2105000 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f0774ff9640 1 -- 172.21.15.46:0/773483370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0778004100 con 0x55f5d2105000 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f0774ff9640 1 -- 172.21.15.46:0/773483370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0778004470 con 0x55f5d2105000 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- 172.21.15.46:0/773483370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2105000 msgr2=0x55f5d20dd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2105000 0x55f5d20dd040 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f07780c4040 tx=0x7f07780b5000 comp rx=0 tx=0).stop 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- 172.21.15.46:0/773483370 shutdown_connections 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2103de0 0x55f5d20dd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d2105000 0x55f5d20dd040 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 --2- 172.21.15.46:0/773483370 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d210a580 0x55f5d210f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- 172.21.15.46:0/773483370 >> 172.21.15.46:0/773483370 conn(0x55f5d20d5a00 msgr2=0x55f5d21104a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- 172.21.15.46:0/773483370 shutdown_connections 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- 172.21.15.46:0/773483370 wait complete. 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- start start 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 Processor -- start 2023-12-16T21:43:29.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d213f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2105000 0x55f5d213fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d210a580 0x55f5d2140170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1f27400 con 0x55f5d210a580 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1f25560 con 0x55f5d2103de0 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f5d1f2c750 con 0x55f5d2105000 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f078fb36000 10 client.0 _ll_register_callbacks cb 0x55f5d1eac510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f0786d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d210a580 0x55f5d2140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f0786d9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d210a580 0x55f5d2140170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50120/0 (socket says 172.21.15.46:50120) 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.439+0000 7f0786d9d640 1 -- 172.21.15.46:0/1432951997 learned_addr learned my addr 172.21.15.46:0/1432951997 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f0785d9b640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2105000 0x55f5d213fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078659c640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d213f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f0786d9d640 1 -- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2105000 msgr2=0x55f5d213fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f0786d9d640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2105000 0x55f5d213fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f0786d9d640 1 -- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 msgr2=0x55f5d213f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f0786d9d640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d213f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f0786d9d640 1 -- 172.21.15.46:0/1432951997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f5d1c21890 con 0x55f5d210a580 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078659c640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d213f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f0786d9d640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d210a580 0x55f5d2140170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f077c001ec0 tx=0x7f077c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaae427182620593 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 1 -- 172.21.15.46:0/1432951997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c014680 con 0x55f5d210a580 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 1 -- 172.21.15.46:0/1432951997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f077c0147e0 con 0x55f5d210a580 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5d1eb0a50 con 0x55f5d210a580 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 1 -- 172.21.15.46:0/1432951997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7f077c004040 con 0x55f5d210a580 2023-12-16T21:43:29.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 10 client.5055 fetch_fsmap learned FSMap version 13 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55f5d1c21cc0 con 0x55f5d210a580 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 1 -- 172.21.15.46:0/1432951997 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f5d1eb0a50 con 0x55f5d210a580 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 1 -- 172.21.15.46:0/1432951997 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 13) v1 ==== 2144+0+0 (secure 0 0 0) 0x7f077c0041a0 con 0x55f5d210a580 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 10 client.5055 fetch_fsmap finished waiting for FSMap version 13 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 10 client.5055 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 20 client.5055 populate_metadata read hostname 'smithi046' 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d2104700 0x55f5d2176e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 4 client.5055 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "status"}] 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 --> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f5d1f57f30 con 0x55f5d2104700 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078659c640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d2104700 0x55f5d2176e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 10 client.5055 ms_handle_connect on v2:172.21.15.46:6832/3724256427 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078659c640 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d2104700 0x55f5d2176e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec01232e4d587983 server_cookie=17f5ea3bc3f2c393 in_seq=0 out_seq=0 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 1 -- 172.21.15.46:0/1432951997 <== mds.0 v2:172.21.15.46:6832/3724256427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55f5d1f57f30 con 0x55f5d2104700 2023-12-16T21:43:29.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 10 client.5055 handle_command_reply: tid=0 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 client.5055 shutdown 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d210a580 msgr2=0x55f5d2140170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d210a580 0x55f5d2140170 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f077c001ec0 tx=0x7f077c0046d0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 shutdown_connections 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:6832/3724256427,v1:172.21.15.46:6833/3724256427] conn(0x55f5d2104700 0x55f5d2176e90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f5d210a580 0x55f5d2140170 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x55f5d2105000 0x55f5d213fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f076d7fa640 0 client.5055 ms_handle_reset on v2:172.21.15.46:6832/3724256427 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 --2- 172.21.15.46:0/1432951997 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x55f5d2103de0 0x55f5d213f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 >> 172.21.15.46:0/1432951997 conn(0x55f5d20d5a00 msgr2=0x55f5d2110d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 shutdown_connections 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 1 -- 172.21.15.46:0/1432951997 wait complete. 2023-12-16T21:43:29.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078fb36000 20 client.5055 trim_cache size 0 max 16384 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "52c183c6-9df9-436e-b409-f347731c6c1e", 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 4863, 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 9.1246607120000007, 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 13, 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 30, 2023-12-16T21:43:29.453 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 30, 2023-12-16T21:43:29.454 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 16.260069124000001 2023-12-16T21:43:29.454 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:29.454 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f07680664a0 msgr2=0x7f0768068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f07680664a0 0x7f0768068960 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07780ae000 tx=0x7f07780a6040 comp rx=0 tx=0).stop 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880bc8f0 msgr2=0x7f0788149830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880bc8f0 0x7f0788149830 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f078000cfc0 tx=0x7f078000a7d0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 shutdown_connections 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f07680664a0 0x7f0768068960 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0788146720 0x7f07881e36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f07881401d0 0x7f07881e3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 --2- 172.21.15.46:0/650957939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f07880bc8f0 0x7f0788149830 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 >> 172.21.15.46:0/650957939 conn(0x7f07880722b0 msgr2=0x7f07880bbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 shutdown_connections 2023-12-16T21:43:29.455 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.443+0000 7f078ca05640 1 -- 172.21.15.46:0/650957939 wait complete. 2023-12-16T21:43:29.466 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-16T21:43:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.575+0000 7fb72712b640 1 Processor -- start 2023-12-16T21:43:29.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.575+0000 7fb72712b640 1 -- start start 2023-12-16T21:43:29.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb72712b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb720071cd0 0x7fb7200b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb72712b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7200b2990 0x7fb7200b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.582 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb72712b640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7200720b0 con 0x7fb720144740 2023-12-16T21:43:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb72712b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72013eca0 con 0x7fb7200b2990 2023-12-16T21:43:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb72712b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200b4c90 con 0x7fb720071cd0 2023-12-16T21:43:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7200b2990 0x7fb7200b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7200b2990 0x7fb7200b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50136/0 (socket says 172.21.15.46:50136) 2023-12-16T21:43:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 -- 172.21.15.46:0/2380254742 learned_addr learned my addr 172.21.15.46:0/2380254742 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.583 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb724ec7640 1 --2- 172.21.15.46:0/2380254742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb720071cd0 0x7fb7200b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 -- 172.21.15.46:0/2380254742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb720071cd0 msgr2=0x7fb7200b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 --2- 172.21.15.46:0/2380254742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb720071cd0 0x7fb7200b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 -- 172.21.15.46:0/2380254742 >> v1:172.21.15.161:6789/0 conn(0x7fb720144740 legacy=0x7fb720144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:29.584 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 -- 172.21.15.46:0/2380254742 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200b4f70 con 0x7fb7200b2990 2023-12-16T21:43:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71ffff640 1 --2- 172.21.15.46:0/2380254742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7200b2990 0x7fb7200b2e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb708009870 tx=0x7fb70801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116b7153840cd028 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71f7fe640 1 -- 172.21.15.46:0/2380254742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080ad640 con 0x7fb7200b2990 2023-12-16T21:43:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71f7fe640 1 -- 172.21.15.46:0/2380254742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7080ad7a0 con 0x7fb7200b2990 2023-12-16T21:43:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.579+0000 7fb71f7fe640 1 -- 172.21.15.46:0/2380254742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080adb30 con 0x7fb7200b2990 2023-12-16T21:43:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- 172.21.15.46:0/2380254742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7200b2990 msgr2=0x7fb7200b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 --2- 172.21.15.46:0/2380254742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7200b2990 0x7fb7200b2e10 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb708009870 tx=0x7fb70801c440 comp rx=0 tx=0).stop 2023-12-16T21:43:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- 172.21.15.46:0/2380254742 shutdown_connections 2023-12-16T21:43:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 --2- 172.21.15.46:0/2380254742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7200b2990 0x7fb7200b2e10 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 --2- 172.21.15.46:0/2380254742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb720071cd0 0x7fb7200b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- 172.21.15.46:0/2380254742 >> 172.21.15.46:0/2380254742 conn(0x7fb7200b5850 msgr2=0x7fb7200b5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- 172.21.15.46:0/2380254742 shutdown_connections 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- 172.21.15.46:0/2380254742 wait complete. 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 Processor -- start 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- start start 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb720071cd0 0x7fb7201dfe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7200b2990 0x7fb72012da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144740 0x7fb72012dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201e0700 con 0x7fb720144740 2023-12-16T21:43:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7201e09e0 con 0x7fb720071cd0 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb720134830 con 0x7fb7200b2990 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144740 0x7fb72012dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144740 0x7fb72012dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50152/0 (socket says 172.21.15.46:50152) 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 -- 172.21.15.46:0/3557717222 learned_addr learned my addr 172.21.15.46:0/3557717222 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb71ffff640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7200b2990 0x7fb72012da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb724ec7640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb720071cd0 0x7fb7201dfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 -- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7200b2990 msgr2=0x7fb72012da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7200b2990 0x7fb72012da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 -- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb720071cd0 msgr2=0x7fb7201dfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.592 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb720071cd0 0x7fb7201dfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 -- 172.21.15.46:0/3557717222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7080094d0 con 0x7fb720144740 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb724ec7640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb720071cd0 0x7fb7201dfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb71ffff640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7200b2990 0x7fb72012da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb7256c8640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144740 0x7fb72012dfd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb71800b410 tx=0x7fb71800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb71d7fa640 1 -- 172.21.15.46:0/3557717222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718011830 con 0x7fb720144740 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb71d7fa640 1 -- 172.21.15.46:0/3557717222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7180119d0 con 0x7fb720144740 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72012c0f0 con 0x7fb720144740 2023-12-16T21:43:29.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb71d7fa640 1 -- 172.21.15.46:0/3557717222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718011d80 con 0x7fb720144740 2023-12-16T21:43:29.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.583+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7201e0360 con 0x7fb720144740 2023-12-16T21:43:29.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.587+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72012d7b0 con 0x7fb720144740 2023-12-16T21:43:29.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.587+0000 7fb71d7fa640 1 -- 172.21.15.46:0/3557717222 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fb7180193b0 con 0x7fb720144740 2023-12-16T21:43:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.587+0000 7fb71d7fa640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb700066470 0x7fb700068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.587+0000 7fb71d7fa640 1 -- 172.21.15.46:0/3557717222 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb718083c20 con 0x7fb720144740 2023-12-16T21:43:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.587+0000 7fb724ec7640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb700066470 0x7fb700068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.587+0000 7fb724ec7640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb700066470 0x7fb700068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb710007330 tx=0x7fb71000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.591+0000 7fb71d7fa640 1 -- 172.21.15.46:0/3557717222 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fb72012d7b0 con 0x7fb720144740 2023-12-16T21:43:29.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.779+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb72012d7b0 con 0x7fb720144740 2023-12-16T21:43:29.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.779+0000 7fb71d7fa640 1 -- 172.21.15.46:0/3557717222 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb72012d7b0 con 0x7fb720144740 2023-12-16T21:43:29.784 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-16T21:43:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb700066470 msgr2=0x7fb700068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb700066470 0x7fb700068930 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb710007330 tx=0x7fb71000b040 comp rx=0 tx=0).stop 2023-12-16T21:43:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144740 msgr2=0x7fb72012dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144740 0x7fb72012dfd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb71800b410 tx=0x7fb71800b8d0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 shutdown_connections 2023-12-16T21:43:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb700066470 0x7fb700068930 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb720144740 0x7fb72012dfd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7200b2990 0x7fb72012da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 --2- 172.21.15.46:0/3557717222 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb720071cd0 0x7fb7201dfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 >> 172.21.15.46:0/3557717222 conn(0x7fb7200b5850 msgr2=0x7fb72006d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 shutdown_connections 2023-12-16T21:43:29.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:29.783+0000 7fb72712b640 1 -- 172.21.15.46:0/3557717222 wait complete. 2023-12-16T21:43:29.803 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-16T21:43:29.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-16T21:43:29.803 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2023-12-16T21:43:29.803 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-16T21:43:29.803 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:29.803 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-16T21:43:29.803 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-16T21:43:29.804 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-16T21:43:29.804 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-16T21:43:29.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:29.804 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-16T21:43:29.810 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42413sec preferred_lft 42413sec 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 02:42:9b:f9:01:fd brd ff:ff:ff:ff:ff:ff 2023-12-16T21:43:29.811 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2023-12-16T21:43:29.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:29.813 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-16T21:43:29.863 INFO:teuthology.orchestra.run.smithi046.stdout:1 2023-12-16T21:43:29.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:29.865 DEBUG:teuthology.orchestra.run.smithi046:> route 2023-12-16T21:43:29.915 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2023-12-16T21:43:29.915 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-16T21:43:29.915 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-16T21:43:29.915 INFO:teuthology.orchestra.run.smithi046.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-16T21:43:29.916 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-16T21:43:29.916 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-16T21:43:29.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:29.916 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec bash -c ' 2023-12-16T21:43:29.916 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2023-12-16T21:43:29.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-16T21:43:29.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-16T21:43:29.916 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-16T21:43:29.916 DEBUG:teuthology.orchestra.run.smithi046:> ') 2023-12-16T21:43:29.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:29.999 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-16T21:43:30.046 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-16T21:43:30.046 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-16T21:43:30.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:30.046 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2023-12-16T21:43:30.103 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-16T21:43:30.103 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-16T21:43:30.103 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:30.150 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-16T21:43:30.150 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-16T21:43:30.150 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:30.197 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-16T21:43:30.255 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-16T21:43:30.304 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-16T21:43:30.304 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-16T21:43:30.305 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-16T21:43:30.306 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-16T21:43:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-16T21:43:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:30.307 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:30.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:30.308 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-16T21:43:30.354 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-16T21:43:30.354 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-16T21:43:30.398 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-16T21:43:30.415 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-16T21:43:30.462 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-16T21:43:30.463 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-16T21:43:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-16T21:43:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-16T21:43:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-16T21:43:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:30.464 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:30.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:30.465 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-16T21:43:30.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:2023-12-16T21:43:30.479+0000 7fadf3004580 -1 init, newargv = 0x558998b9d940 newargc=15 2023-12-16T21:43:30.484 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[46609]: starting ceph client 2023-12-16T21:43:30.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[46609]: starting fuse 2023-12-16T21:43:31.591 DEBUG:teuthology.orchestra.run.smithi046:> sudo modprobe fuse 2023-12-16T21:43:31.607 DEBUG:teuthology.orchestra.run.smithi046:> cat /proc/self/mounts | awk '{print $2}' 2023-12-16T21:43:31.656 INFO:teuthology.orchestra.run.smithi046.stdout:/sys 2023-12-16T21:43:31.656 INFO:teuthology.orchestra.run.smithi046.stdout:/proc 2023-12-16T21:43:31.656 INFO:teuthology.orchestra.run.smithi046.stdout:/dev 2023-12-16T21:43:31.656 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/pts 2023-12-16T21:43:31.656 INFO:teuthology.orchestra.run.smithi046.stdout:/run 2023-12-16T21:43:31.656 INFO:teuthology.orchestra.run.smithi046.stdout:/ 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/security 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/shm 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/run/lock 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/cgroup 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/pstore 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/bpf 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/sys/fs/binfmt_misc 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/hugepages 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/dev/mqueue 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/debug 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/tracing 2023-12-16T21:43:31.657 INFO:teuthology.orchestra.run.smithi046.stdout:/proc/fs/nfsd 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/fs/fuse/connections 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/sys/kernel/config 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/run/credentials/systemd-sysusers.service 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/run/rpc_pipefs 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/run/user/1000 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-0 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-1 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-2 2023-12-16T21:43:31.658 INFO:teuthology.orchestra.run.smithi046.stdout:/var/lib/ceph/osd/ceph-3 2023-12-16T21:43:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns 2023-12-16T21:43:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:31.659 INFO:teuthology.orchestra.run.smithi046.stdout:/home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:31.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:31.660 DEBUG:teuthology.orchestra.run.smithi046:> ls /sys/fs/fuse/connections 2023-12-16T21:43:31.705 INFO:teuthology.orchestra.run.smithi046.stdout:45 2023-12-16T21:43:31.706 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-12-16T21:43:31.706 DEBUG:teuthology.orchestra.run.smithi046:> sudo stdin-killer -- python3 -c ' 2023-12-16T21:43:31.706 DEBUG:teuthology.orchestra.run.smithi046:> import glob 2023-12-16T21:43:31.706 DEBUG:teuthology.orchestra.run.smithi046:> import re 2023-12-16T21:43:31.706 DEBUG:teuthology.orchestra.run.smithi046:> import os 2023-12-16T21:43:31.706 DEBUG:teuthology.orchestra.run.smithi046:> import subprocess 2023-12-16T21:43:31.706 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:43:31.706 DEBUG:teuthology.orchestra.run.smithi046:> def _find_admin_socket(client_name): 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> files = glob.glob(asok_path) 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> # Given a non-glob path, it better be there 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> if "*" not in asok_path: 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> assert(len(files) == 1) 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> return files[0] 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> for f in files: 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-16T21:43:31.707 DEBUG:teuthology.orchestra.run.smithi046:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-16T21:43:31.708 DEBUG:teuthology.orchestra.run.smithi046:> contents = proc_f.read() 2023-12-16T21:43:31.708 DEBUG:teuthology.orchestra.run.smithi046:> if mountpoint in contents: 2023-12-16T21:43:31.708 DEBUG:teuthology.orchestra.run.smithi046:> return f 2023-12-16T21:43:31.708 DEBUG:teuthology.orchestra.run.smithi046:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-16T21:43:31.708 DEBUG:teuthology.orchestra.run.smithi046:> 2023-12-16T21:43:31.708 DEBUG:teuthology.orchestra.run.smithi046:> print(_find_admin_socket("client.0")) 2023-12-16T21:43:31.708 DEBUG:teuthology.orchestra.run.smithi046:> ' 2023-12-16T21:43:31.833 INFO:teuthology.orchestra.run.smithi046.stdout:/var/run/ceph/ceph-client.0.46609.asok 2023-12-16T21:43:31.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-12-16T21:43:31.845 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46609.asok 2023-12-16T21:43:31.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:31.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46609.asok status 2023-12-16T21:43:32.004 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "metadata": { 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_sha1": "c7586e95519b7be76f3b6e3a4b333888baf46bc9", 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-202-gc7586e95 (c7586e95519b7be76f3b6e3a4b333888baf46bc9) reef (dev)", 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "entity_id": "0", 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": "smithi046", 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "pid": "46609", 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "root": "/" 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:32.005 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_count": 0, 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "dentry_pinned_count": 0, 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 5067, 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "inst": { 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "name": { 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "client", 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "num": 5067 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4024202883 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: } 2023-12-16T21:43:32.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": { 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "192.168.0.1:0", 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4024202883 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "inst_str": "client.5067 192.168.0.1:0/4024202883", 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "addr_str": "192.168.0.1:0/4024202883", 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "inode_count": 1, 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_epoch": 13, 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch": 30, 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_epoch_barrier": 0, 2023-12-16T21:43:32.007 INFO:teuthology.orchestra.run.smithi046.stdout: "blocklisted": false, 2023-12-16T21:43:32.008 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs" 2023-12-16T21:43:32.008 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:32.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:32.018 DEBUG:teuthology.orchestra.run.smithi046:> stat --file-system '--printf=%T 2023-12-16T21:43:32.019 DEBUG:teuthology.orchestra.run.smithi046:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:32.025 INFO:teuthology.orchestra.run.smithi046.stdout:fuseblk 2023-12-16T21:43:32.026 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:32.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:32.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:32.082 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-12-16T21:43:32.129 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.oBhpdpnOHg 2023-12-16T21:43:32.129 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:43:32.129 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/tmp/tmp.oBhpdpnOHg 2023-12-16T21:43:32.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:32.177 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p dir1/dir2/dir3') 2023-12-16T21:43:32.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:32.303 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'chmod 755 /tmp/test-cephfsynu_kybc') 2023-12-16T21:43:32.312 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfsynu_kybc" 2023-12-16T21:43:32.312 DEBUG:teuthology.orchestra.run.smithi046:> cephfs-shell -b /tmp/test-cephfsynu_kybc 2023-12-16T21:43:32.700 INFO:teuthology.orchestra.run.smithi046.stderr:Unrecognized command 2023-12-16T21:43:32.731 DEBUG:teuthology.orchestra.run:got remote process result: 127 2023-12-16T21:43:32.732 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-16T21:43:32.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-16T21:43:32.778 DEBUG:teuthology.orchestra.run.smithi161:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-16T21:43:32.823 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-16T21:43:32.823 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:43:32.823 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-16T21:43:32.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:33.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883b283640 1 Processor -- start 2023-12-16T21:43:33.005 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883b283640 1 -- start start 2023-12-16T21:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883b283640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834148500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883b283640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8834145350 0x7f883414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883b283640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8834148070 con 0x7f8834072190 2023-12-16T21:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883b283640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88340b10d0 con 0x7f8834145350 2023-12-16T21:43:33.006 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883b283640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883406cd00 con 0x7f883406c920 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f8839820640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8834145350 0x7f883414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883901f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834148500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883901f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834148500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:52710/0 (socket says 172.21.15.46:52710) 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f8839820640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8834145350 0x7f883414eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:51388/0 (socket says 172.21.15.46:51388) 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883901f640 1 -- 172.21.15.46:0/2829881742 learned_addr learned my addr 172.21.15.46:0/2829881742 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883901f640 1 -- 172.21.15.46:0/2829881742 >> v2:172.21.15.161:3300/0 conn(0x7f8834145350 msgr2=0x7f883414eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883901f640 1 --2- 172.21.15.46:0/2829881742 >> v2:172.21.15.161:3300/0 conn(0x7f8834145350 0x7f883414eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883901f640 1 -- 172.21.15.46:0/2829881742 >> v1:172.21.15.161:6789/0 conn(0x7f8834072190 legacy=0x7f8834144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f883901f640 1 -- 172.21.15.46:0/2829881742 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8834148a40 con 0x7f883406c920 2023-12-16T21:43:33.007 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.003+0000 7f8839820640 1 --2- 172.21.15.46:0/2829881742 >> v2:172.21.15.161:3300/0 conn(0x7f8834145350 0x7f883414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:33.009 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883901f640 1 --2- 172.21.15.46:0/2829881742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834148500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8828009870 tx=0x7f882801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef5ac36fa2425462 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8827fff640 1 -- 172.21.15.46:0/2829881742 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280a6040 con 0x7f883406c920 2023-12-16T21:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8827fff640 1 -- 172.21.15.46:0/2829881742 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88280aa600 con 0x7f883406c920 2023-12-16T21:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8827fff640 1 -- 172.21.15.46:0/2829881742 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88280aa990 con 0x7f883406c920 2023-12-16T21:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- 172.21.15.46:0/2829881742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 msgr2=0x7f8834148500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.010 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 --2- 172.21.15.46:0/2829881742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834148500 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8828009870 tx=0x7f882801c480 comp rx=0 tx=0).stop 2023-12-16T21:43:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- 172.21.15.46:0/2829881742 shutdown_connections 2023-12-16T21:43:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 --2- 172.21.15.46:0/2829881742 >> v2:172.21.15.161:3300/0 conn(0x7f8834145350 0x7f883414eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 --2- 172.21.15.46:0/2829881742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834148500 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- 172.21.15.46:0/2829881742 >> 172.21.15.46:0/2829881742 conn(0x7f88340b48e0 msgr2=0x7f88340b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- 172.21.15.46:0/2829881742 shutdown_connections 2023-12-16T21:43:33.011 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- 172.21.15.46:0/2829881742 wait complete. 2023-12-16T21:43:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 Processor -- start 2023-12-16T21:43:33.013 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- start start 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8834145350 0x7f8834137870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883413d4a0 0x7f88341354e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8834146de0 con 0x7f883413d4a0 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8834146a60 con 0x7f8834145350 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883b283640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8834145b50 con 0x7f883406c920 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883413d4a0 0x7f88341354e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883413d4a0 0x7f88341354e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50176/0 (socket says 172.21.15.46:50176) 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 -- 172.21.15.46:0/1845284136 learned_addr learned my addr 172.21.15.46:0/1845284136 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:33.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883901f640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883881e640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8834145350 0x7f8834137870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 -- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 msgr2=0x7f8834146440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834146440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 -- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8834145350 msgr2=0x7f8834137870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8834145350 0x7f8834137870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f8839820640 1 -- 172.21.15.46:0/1845284136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c009420 con 0x7f883413d4a0 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.007+0000 7f883901f640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f8839820640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883413d4a0 0x7f88341354e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f882c00e840 tx=0x7f882c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f8825ffb640 1 -- 172.21.15.46:0/1845284136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00c940 con 0x7f883413d4a0 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f8825ffb640 1 -- 172.21.15.46:0/1845284136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f882c00cac0 con 0x7f883413d4a0 2023-12-16T21:43:33.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88280094d0 con 0x7f883413d4a0 2023-12-16T21:43:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f8825ffb640 1 -- 172.21.15.46:0/1845284136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c00ce70 con 0x7f883413d4a0 2023-12-16T21:43:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88341460d0 con 0x7f883413d4a0 2023-12-16T21:43:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88340679e0 con 0x7f883413d4a0 2023-12-16T21:43:33.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f8825ffb640 1 -- 172.21.15.46:0/1845284136 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f882c013470 con 0x7f883413d4a0 2023-12-16T21:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f8825ffb640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f88180664a0 0x7f8818068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f8825ffb640 1 -- 172.21.15.46:0/1845284136 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f882c0843e0 con 0x7f883413d4a0 2023-12-16T21:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f883901f640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f88180664a0 0x7f8818068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.011+0000 7f883901f640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f88180664a0 0x7f8818068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88280ae000 tx=0x7f88280a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.015+0000 7f8825ffb640 1 -- 172.21.15.46:0/1845284136 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f882c051850 con 0x7f883413d4a0 2023-12-16T21:43:33.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.175+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88340b1530 con 0x7f883413d4a0 2023-12-16T21:43:33.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.175+0000 7f8825ffb640 1 -- 172.21.15.46:0/1845284136 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f882c054e70 con 0x7f883413d4a0 2023-12-16T21:43:33.179 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:33.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:21.267009+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:43:17.887506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-16T21:43:18.612695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f88180664a0 msgr2=0x7f8818068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f88180664a0 0x7f8818068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f88280ae000 tx=0x7f88280a7000 comp rx=0 tx=0).stop 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883413d4a0 msgr2=0x7f88341354e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883413d4a0 0x7f88341354e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f882c00e840 tx=0x7f882c00ed00 comp rx=0 tx=0).stop 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 shutdown_connections 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f88180664a0 0x7f8818068960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f883413d4a0 0x7f88341354e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8834145350 0x7f8834137870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 --2- 172.21.15.46:0/1845284136 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f883406c920 0x7f8834146440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 >> 172.21.15.46:0/1845284136 conn(0x7f88340b48e0 msgr2=0x7f8834143000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 shutdown_connections 2023-12-16T21:43:33.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.179+0000 7f883b283640 1 -- 172.21.15.46:0/1845284136 wait complete. 2023-12-16T21:43:33.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:43:17.887506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:43:18.612695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:43:33.199 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:43:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980e569640 1 Processor -- start 2023-12-16T21:43:33.315 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980e569640 1 -- start start 2023-12-16T21:43:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980e569640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98080af300 0x7f9808144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980e569640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 0x7f9808144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980e569640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98080b1730 con 0x7f98080b1eb0 2023-12-16T21:43:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980e569640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f980806cd70 con 0x7f98080af300 2023-12-16T21:43:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980e569640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98080b0ad0 con 0x7f98080b4350 2023-12-16T21:43:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f9807fff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98080b1eb0 0x7f98080aff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56076/0 (socket says 172.21.15.46:56076) 2023-12-16T21:43:33.316 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f9807fff640 1 -- 172.21.15.46:0/2092915560 learned_addr learned my addr 172.21.15.46:0/2092915560 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f98077fe640 1 --2- 172.21.15.46:0/2092915560 >> v2:172.21.15.161:3300/0 conn(0x7f98080af300 0x7f9808144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f980cb06640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 0x7f9808144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.311+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49917737 0 0) 0x7f98080b1730 con 0x7f98080b1eb0 2023-12-16T21:43:33.318 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f4003530 con 0x7f98080b1eb0 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874325440 0 0) 0x7f97f4003530 con 0x7f98080b1eb0 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 msgr2=0x7f9808144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 0x7f9808144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 >> v2:172.21.15.161:3300/0 conn(0x7f98080af300 msgr2=0x7f9808144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 --2- 172.21.15.46:0/2092915560 >> v2:172.21.15.161:3300/0 conn(0x7f98080af300 0x7f9808144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980806c7f0 con 0x7f98080b1eb0 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f9800001a00 con 0x7f98080b1eb0 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 >> v1:172.21.15.46:6789/0 conn(0x7f98080b1eb0 legacy=0x7f98080aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 0x7f97f4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4003f00 0x7f97f40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4004d10 0x7f97f4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980806cd70 con 0x7f97f4003f00 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f400b9f0 con 0x7f97f4004d10 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f400bbb0 con 0x7f98080b4350 2023-12-16T21:43:33.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f98077fe640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4003f00 0x7f97f40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9807fff640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 0x7f97f4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f980cb06640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4004d10 0x7f97f4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f98077fe640 1 -- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 msgr2=0x7f97f4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f98077fe640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98080b4350 0x7f97f4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f98077fe640 1 -- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4004d10 msgr2=0x7f97f4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f98077fe640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f97f4004d10 0x7f97f4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f98077fe640 1 -- 172.21.15.46:0/2092915560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4003530 con 0x7f97f4003f00 2023-12-16T21:43:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f98077fe640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4003f00 0x7f97f40047d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009ae0 tx=0x7f97f001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22c192871a17119 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00a5570 con 0x7f97f4003f00 2023-12-16T21:43:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f00a56d0 con 0x7f97f4003f00 2023-12-16T21:43:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.315+0000 7f9806ffd640 1 -- 172.21.15.46:0/2092915560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00a5a80 con 0x7f97f4003f00 2023-12-16T21:43:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/2092915560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4003f00 msgr2=0x7f97f40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 --2- 172.21.15.46:0/2092915560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97f4003f00 0x7f97f40047d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f97f0009ae0 tx=0x7f97f001ca90 comp rx=0 tx=0).stop 2023-12-16T21:43:33.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980cb06640 1 -- 172.21.15.46:0/2092915560 reap_dead start 2023-12-16T21:43:33.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/2092915560 shutdown_connections 2023-12-16T21:43:33.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/2092915560 >> 172.21.15.46:0/2092915560 conn(0x7f98080b4900 msgr2=0x7f98080b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:33.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/2092915560 shutdown_connections 2023-12-16T21:43:33.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/2092915560 wait complete. 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 Processor -- start 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- start start 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f980809b080 0x7f980809b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98081e0b40 0x7f98081e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98081dcf40 0x7f98081dd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980809db40 con 0x7f98081e0b40 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980809d860 con 0x7f980809b080 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98081e0620 con 0x7f98081dcf40 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98081e0b40 0x7f98081e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98081e0b40 0x7f98081e0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50190/0 (socket says 172.21.15.46:50190) 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 -- 172.21.15.46:0/3236615339 learned_addr learned my addr 172.21.15.46:0/3236615339 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f9807fff640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f980809b080 0x7f980809b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980cb06640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98081dcf40 0x7f98081dd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 -- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98081dcf40 msgr2=0x7f98081dd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98081dcf40 0x7f98081dd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 -- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f980809b080 msgr2=0x7f980809b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f980809b080 0x7f980809b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 -- 172.21.15.46:0/3236615339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98081445f0 con 0x7f98081e0b40 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f98077fe640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98081e0b40 0x7f98081e0fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97f001c8c0 tx=0x7f97f0019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f9804ff9640 1 -- 172.21.15.46:0/3236615339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00a5e60 con 0x7f98081e0b40 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f9804ff9640 1 -- 172.21.15.46:0/3236615339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f0004020 con 0x7f98081e0b40 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980809b9a0 con 0x7f98081e0b40 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f9804ff9640 1 -- 172.21.15.46:0/3236615339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f00043d0 con 0x7f98081e0b40 2023-12-16T21:43:33.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98081dd900 con 0x7f98081e0b40 2023-12-16T21:43:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.319+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98081443f0 con 0x7f98081e0b40 2023-12-16T21:43:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.323+0000 7f9804ff9640 1 -- 172.21.15.46:0/3236615339 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f97f00b4070 con 0x7f98081e0b40 2023-12-16T21:43:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.323+0000 7f9804ff9640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f97e8066470 0x7f97e8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.323+0000 7f9804ff9640 1 -- 172.21.15.46:0/3236615339 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97f011e760 con 0x7f98081e0b40 2023-12-16T21:43:33.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.323+0000 7f9807fff640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f97e8066470 0x7f97e8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.323+0000 7f9807fff640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f97e8066470 0x7f97e8068930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9800001340 tx=0x7f9800009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.327+0000 7f9804ff9640 1 -- 172.21.15.46:0/3236615339 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f97f00ebbd0 con 0x7f98081e0b40 2023-12-16T21:43:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.483+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f980809d650 con 0x7f98081e0b40 2023-12-16T21:43:33.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.483+0000 7f9804ff9640 1 -- 172.21.15.46:0/3236615339 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f97f00b20a0 con 0x7f98081e0b40 2023-12-16T21:43:33.488 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f97e8066470 msgr2=0x7f97e8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f97e8066470 0x7f97e8068930 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9800001340 tx=0x7f9800009040 comp rx=0 tx=0).stop 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98081e0b40 msgr2=0x7f98081e0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98081e0b40 0x7f98081e0fc0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f97f001c8c0 tx=0x7f97f0019830 comp rx=0 tx=0).stop 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 shutdown_connections 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f97e8066470 0x7f97e8068930 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98081dcf40 0x7f98081dd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98081e0b40 0x7f98081e0fc0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 --2- 172.21.15.46:0/3236615339 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f980809b080 0x7f980809b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 >> 172.21.15.46:0/3236615339 conn(0x7f98080b4900 msgr2=0x7f98080b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 shutdown_connections 2023-12-16T21:43:33.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.487+0000 7f980e569640 1 -- 172.21.15.46:0/3236615339 wait complete. 2023-12-16T21:43:33.508 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:43:33.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d7268640 1 Processor -- start 2023-12-16T21:43:33.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d7268640 1 -- start start 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d7268640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fb7d00bc8a0 0x7fb7d0149e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d7268640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7d00b6fc0 0x7fb7d0151200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d7268640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d00b4990 con 0x7fb7d014a350 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d7268640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d00b5de0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d7268640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d00b5b70 con 0x7fb7d00bc8a0 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d4803640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fb7d014a350 0x7fb7d00b68b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56082/0 (socket says 172.21.15.46:56082) 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d5805640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fb7d00b6fc0 0x7fb7d0151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d4803640 1 -- 172.21.15.46:0/2722740731 learned_addr learned my addr 172.21.15.46:0/2722740731 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d5004640 1 --2- 172.21.15.46:0/2722740731 >> v2:172.21.15.161:3300/0 conn(0x7fb7d00bc8a0 0x7fb7d0149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d5805640 1 -- 172.21.15.46:0/2722740731 >> v2:172.21.15.161:3300/0 conn(0x7fb7d00bc8a0 msgr2=0x7fb7d0149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d5805640 1 --2- 172.21.15.46:0/2722740731 >> v2:172.21.15.161:3300/0 conn(0x7fb7d00bc8a0 0x7fb7d0149e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d5805640 1 -- 172.21.15.46:0/2722740731 >> v1:172.21.15.46:6789/0 conn(0x7fb7d014a350 legacy=0x7fb7d00b68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.635+0000 7fb7d5805640 1 -- 172.21.15.46:0/2722740731 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d014a7e0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5805640 1 --2- 172.21.15.46:0/2722740731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0151200 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009d90 tx=0x7fb7c801c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce79a06d6c43cc6f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7c7fff640 1 -- 172.21.15.46:0/2722740731 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80a58b0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7c7fff640 1 -- 172.21.15.46:0/2722740731 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c80a5a10 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7c7fff640 1 -- 172.21.15.46:0/2722740731 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c80a5da0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- 172.21.15.46:0/2722740731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 msgr2=0x7fb7d0151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 --2- 172.21.15.46:0/2722740731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0151200 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb7c8009d90 tx=0x7fb7c801c7f0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- 172.21.15.46:0/2722740731 shutdown_connections 2023-12-16T21:43:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 --2- 172.21.15.46:0/2722740731 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0151200 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 --2- 172.21.15.46:0/2722740731 >> v2:172.21.15.161:3300/0 conn(0x7fb7d00bc8a0 0x7fb7d0149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.642 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- 172.21.15.46:0/2722740731 >> 172.21.15.46:0/2722740731 conn(0x7fb7d00722b0 msgr2=0x7fb7d00bbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- 172.21.15.46:0/2722740731 shutdown_connections 2023-12-16T21:43:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- 172.21.15.46:0/2722740731 wait complete. 2023-12-16T21:43:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 Processor -- start 2023-12-16T21:43:33.643 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- start start 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0137810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7d00bc8a0 0x7fb7d013b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d014a350 0x7fb7d013b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d01dfda0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d0137d50 con 0x7fb7d014a350 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d7268640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d0138aa0 con 0x7fb7d00bc8a0 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50212/0 (socket says 172.21.15.46:50212) 2023-12-16T21:43:33.645 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 -- 172.21.15.46:0/3698655720 learned_addr learned my addr 172.21.15.46:0/3698655720 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5805640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d014a350 0x7fb7d013b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d4803640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7d00bc8a0 0x7fb7d013b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 -- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7d00bc8a0 msgr2=0x7fb7d013b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7d00bc8a0 0x7fb7d013b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 -- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d014a350 msgr2=0x7fb7d013b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d014a350 0x7fb7d013b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.639+0000 7fb7d5004640 1 -- 172.21.15.46:0/3698655720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c8009a70 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7d5004640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0137810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00d830 tx=0x7fb7cc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7c5ffb640 1 -- 172.21.15.46:0/3698655720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00b6c0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7c5ffb640 1 -- 172.21.15.46:0/3698655720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7cc00b820 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d013fbe0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7c5ffb640 1 -- 172.21.15.46:0/3698655720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc00bb90 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.646 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d01df9c0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d0068960 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7c5ffb640 1 -- 172.21.15.46:0/3698655720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fb7cc01e460 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.647 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7c5ffb640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb7b4066470 0x7fb7b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7c5ffb640 1 -- 172.21.15.46:0/3698655720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7cc0845d0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7d4803640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb7b4066470 0x7fb7b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.648 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.643+0000 7fb7d4803640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb7b4066470 0x7fb7b4068930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb7d013f910 tx=0x7fb7c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.647+0000 7fb7c5ffb640 1 -- 172.21.15.46:0/3698655720 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fb7cc051ae0 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.803+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7d0144020 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.803+0000 7fb7c5ffb640 1 -- 172.21.15.46:0/3698655720 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7cc055100 con 0x7fb7d00b6fc0 2023-12-16T21:43:33.807 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb7b4066470 msgr2=0x7fb7b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb7b4066470 0x7fb7b4068930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb7d013f910 tx=0x7fb7c0009040 comp rx=0 tx=0).stop 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 msgr2=0x7fb7d0137810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0137810 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00d830 tx=0x7fb7cc00dcf0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 shutdown_connections 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fb7b4066470 0x7fb7b4068930 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fb7d014a350 0x7fb7d013b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fb7d00bc8a0 0x7fb7d013b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 --2- 172.21.15.46:0/3698655720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7d00b6fc0 0x7fb7d0137810 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 >> 172.21.15.46:0/3698655720 conn(0x7fb7d00722b0 msgr2=0x7fb7d0150f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:33.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 shutdown_connections 2023-12-16T21:43:33.811 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.807+0000 7fb7d7268640 1 -- 172.21.15.46:0/3698655720 wait complete. 2023-12-16T21:43:33.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:43:33.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.943+0000 7f1a7591c640 1 Processor -- start 2023-12-16T21:43:33.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.943+0000 7f1a7591c640 1 -- start start 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a7591c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a700b6ed0 0x7f1a7006c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a7591c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1a700b4940 0x7f1a7006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a7591c640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a70140480 0x7f1a70140930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a7591c640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a700b6420 con 0x7f1a70140480 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a7591c640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a700b5e10 con 0x7f1a700b4940 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a7591c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b72b0 con 0x7f1a700b6ed0 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a70140480 0x7f1a70140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1a70140480 0x7f1a70140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50216/0 (socket says 172.21.15.46:50216) 2023-12-16T21:43:33.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 -- 172.21.15.46:0/181295335 learned_addr learned my addr 172.21.15.46:0/181295335 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6effd640 1 --2- 172.21.15.46:0/181295335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a700b6ed0 0x7f1a7006c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 -- 172.21.15.46:0/181295335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a700b6ed0 msgr2=0x7f1a7006c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 --2- 172.21.15.46:0/181295335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a700b6ed0 0x7f1a7006c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 -- 172.21.15.46:0/181295335 >> v2:172.21.15.161:3300/0 conn(0x7f1a700b4940 msgr2=0x7f1a7006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6e7fc640 1 --2- 172.21.15.46:0/181295335 >> v2:172.21.15.161:3300/0 conn(0x7f1a700b4940 0x7f1a7006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 --2- 172.21.15.46:0/181295335 >> v2:172.21.15.161:3300/0 conn(0x7f1a700b4940 0x7f1a7006cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 -- 172.21.15.46:0/181295335 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a700b60b0 con 0x7f1a70140480 2023-12-16T21:43:33.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6f7fe640 1 --2- 172.21.15.46:0/181295335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a70140480 0x7f1a70140930 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1a700b7590 tx=0x7f1a6801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64357440522c3fc server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6dffb640 1 -- 172.21.15.46:0/181295335 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680ab600 con 0x7f1a70140480 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6dffb640 1 -- 172.21.15.46:0/181295335 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a680a7050 con 0x7f1a70140480 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.947+0000 7f1a6dffb640 1 -- 172.21.15.46:0/181295335 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a680ab980 con 0x7f1a70140480 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/181295335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a70140480 msgr2=0x7f1a70140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 --2- 172.21.15.46:0/181295335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a70140480 0x7f1a70140930 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1a700b7590 tx=0x7f1a6801c470 comp rx=0 tx=0).stop 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/181295335 shutdown_connections 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 --2- 172.21.15.46:0/181295335 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a70140480 0x7f1a70140930 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.954 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 --2- 172.21.15.46:0/181295335 >> v2:172.21.15.161:3300/0 conn(0x7f1a700b4940 0x7f1a7006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 --2- 172.21.15.46:0/181295335 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a700b6ed0 0x7f1a7006c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/181295335 >> 172.21.15.46:0/181295335 conn(0x7f1a7006a750 msgr2=0x7f1a7006ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/181295335 shutdown_connections 2023-12-16T21:43:33.955 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/181295335 wait complete. 2023-12-16T21:43:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 Processor -- start 2023-12-16T21:43:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- start start 2023-12-16T21:43:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a700b4940 0x7f1a700b83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a700b6ed0 0x7f1a70142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.957 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a70140480 0x7f1a700b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a70136e20 con 0x7f1a700b6ed0 2023-12-16T21:43:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b8a60 con 0x7f1a700b4940 2023-12-16T21:43:33.958 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700b8d40 con 0x7f1a70140480 2023-12-16T21:43:33.960 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a700b6ed0 0x7f1a70142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a700b6ed0 0x7f1a70142d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50230/0 (socket says 172.21.15.46:50230) 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 -- 172.21.15.46:0/3408186107 learned_addr learned my addr 172.21.15.46:0/3408186107 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6f7fe640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a70140480 0x7f1a700b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6effd640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a700b4940 0x7f1a700b83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 -- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a70140480 msgr2=0x7f1a700b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a70140480 0x7f1a700b37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 -- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a700b4940 msgr2=0x7f1a700b83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a700b4940 0x7f1a700b83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 -- 172.21.15.46:0/3408186107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680094d0 con 0x7f1a700b6ed0 2023-12-16T21:43:33.961 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a6e7fc640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a700b6ed0 0x7f1a70142d20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7f0 tx=0x7f1a5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a67fff640 1 -- 172.21.15.46:0/3408186107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004220 con 0x7f1a700b6ed0 2023-12-16T21:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a67fff640 1 -- 172.21.15.46:0/3408186107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58004380 con 0x7f1a700b6ed0 2023-12-16T21:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a700b2cb0 con 0x7f1a700b6ed0 2023-12-16T21:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a67fff640 1 -- 172.21.15.46:0/3408186107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58003840 con 0x7f1a700b6ed0 2023-12-16T21:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a70143970 con 0x7f1a700b6ed0 2023-12-16T21:43:33.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.951+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a700b2b20 con 0x7f1a700b6ed0 2023-12-16T21:43:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.955+0000 7f1a67fff640 1 -- 172.21.15.46:0/3408186107 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f1a58003bb0 con 0x7f1a700b6ed0 2023-12-16T21:43:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.955+0000 7f1a67fff640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1a50066470 0x7f1a50068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.955+0000 7f1a67fff640 1 -- 172.21.15.46:0/3408186107 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a58083d80 con 0x7f1a700b6ed0 2023-12-16T21:43:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.955+0000 7f1a6effd640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1a50066470 0x7f1a50068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:33.963 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.955+0000 7f1a6effd640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1a50066470 0x7f1a50068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a700baf20 tx=0x7f1a600083e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:33.964 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:33.959+0000 7f1a67fff640 1 -- 172.21.15.46:0/3408186107 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f1a58051290 con 0x7f1a700b6ed0 2023-12-16T21:43:34.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.111+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1a700b11b0 con 0x7f1a700b6ed0 2023-12-16T21:43:34.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.111+0000 7f1a67fff640 1 -- 172.21.15.46:0/3408186107 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1a5801e0a0 con 0x7f1a700b6ed0 2023-12-16T21:43:34.117 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:43:34.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1a50066470 msgr2=0x7f1a50068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1a50066470 0x7f1a50068930 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1a700baf20 tx=0x7f1a600083e0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a700b6ed0 msgr2=0x7f1a70142d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a700b6ed0 0x7f1a70142d20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7f0 tx=0x7f1a5800ccb0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 shutdown_connections 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1a50066470 0x7f1a50068930 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1a70140480 0x7f1a700b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1a700b6ed0 0x7f1a70142d20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 --2- 172.21.15.46:0/3408186107 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1a700b4940 0x7f1a700b83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 >> 172.21.15.46:0/3408186107 conn(0x7f1a7006a750 msgr2=0x7f1a70148100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 shutdown_connections 2023-12-16T21:43:34.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.115+0000 7f1a7591c640 1 -- 172.21.15.46:0/3408186107 wait complete. 2023-12-16T21:43:34.139 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-16T21:43:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff4a6ed640 1 Processor -- start 2023-12-16T21:43:34.270 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff4a6ed640 1 -- start start 2023-12-16T21:43:34.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff4a6ed640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 0x7eff44146db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff4a6ed640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4406d4f0 con 0x7eff441472f0 2023-12-16T21:43:34.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff4a6ed640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff440b3460 con 0x7eff440736b0 2023-12-16T21:43:34.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff4a6ed640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4406a620 con 0x7eff4406b7a0 2023-12-16T21:43:34.272 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7eff441472f0 0x7eff44151610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56086/0 (socket says 172.21.15.46:56086) 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 -- 172.21.15.46:0/3125197723 learned_addr learned my addr 172.21.15.46:0/3125197723 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff437fe640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 0x7eff44146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969524329 0 0) 0x7eff4406d4f0 con 0x7eff441472f0 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff2c003530 con 0x7eff441472f0 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3080913391 0 0) 0x7eff2c003530 con 0x7eff441472f0 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 msgr2=0x7eff44146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 0x7eff44146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 >> v1:172.21.15.161:6789/0 conn(0x7eff440736b0 legacy=0x7eff4406ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4406a900 con 0x7eff441472f0 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7eff34001680 con 0x7eff441472f0 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 >> v1:172.21.15.46:6789/0 conn(0x7eff441472f0 legacy=0x7eff44151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 0x7eff2c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff2c003f30 0x7eff2c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c004d10 0x7eff2c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4406a620 con 0x7eff2c004d10 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c00bb20 con 0x7eff2c003f30 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2c00bce0 con 0x7eff4406b7a0 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c004d10 0x7eff2c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff43fff640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 0x7eff2c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff437fe640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff2c003f30 0x7eff2c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 -- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 msgr2=0x7eff2c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff4406b7a0 0x7eff2c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 -- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff2c003f30 msgr2=0x7eff2c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff2c003f30 0x7eff2c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.267+0000 7eff48c8a640 1 -- 172.21.15.46:0/3125197723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c003530 con 0x7eff2c004d10 2023-12-16T21:43:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff48c8a640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c004d10 0x7eff2c005190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c240 tx=0x7eff3401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01ad91bc4df29fd server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34005ae0 con 0x7eff2c004d10 2023-12-16T21:43:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff340ab070 con 0x7eff2c004d10 2023-12-16T21:43:34.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff42ffd640 1 -- 172.21.15.46:0/3125197723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34007070 con 0x7eff2c004d10 2023-12-16T21:43:34.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 -- 172.21.15.46:0/3125197723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c004d10 msgr2=0x7eff2c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 --2- 172.21.15.46:0/3125197723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff2c004d10 0x7eff2c005190 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff3400c240 tx=0x7eff3401f3b0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff48c8a640 1 -- 172.21.15.46:0/3125197723 reap_dead start 2023-12-16T21:43:34.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 -- 172.21.15.46:0/3125197723 shutdown_connections 2023-12-16T21:43:34.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 -- 172.21.15.46:0/3125197723 >> 172.21.15.46:0/3125197723 conn(0x7eff440b85e0 msgr2=0x7eff440b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:34.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 -- 172.21.15.46:0/3125197723 shutdown_connections 2023-12-16T21:43:34.276 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 -- 172.21.15.46:0/3125197723 wait complete. 2023-12-16T21:43:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 Processor -- start 2023-12-16T21:43:34.278 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.271+0000 7eff4a6ed640 1 -- start start 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff440b1900 0x7eff440afb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff440b1ce0 0x7eff441dc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff441dc7f0 0x7eff441460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff440b24c0 con 0x7eff440b1900 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff441dbe90 con 0x7eff440b1ce0 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff441dbb80 con 0x7eff441dc7f0 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff440b1900 0x7eff440afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff440b1900 0x7eff440afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50240/0 (socket says 172.21.15.46:50240) 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 -- 172.21.15.46:0/164324103 learned_addr learned my addr 172.21.15.46:0/164324103 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff437fe640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff440b1ce0 0x7eff441dc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff48c8a640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff441dc7f0 0x7eff441460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 -- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff441dc7f0 msgr2=0x7eff441460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff441dc7f0 0x7eff441460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 -- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff440b1ce0 msgr2=0x7eff441dc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff440b1ce0 0x7eff441dc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 -- 172.21.15.46:0/164324103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff44066d70 con 0x7eff440b1900 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff48c8a640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff441dc7f0 0x7eff441460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff43fff640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff440b1900 0x7eff440afb90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00e580 tx=0x7eff3c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff40ff9640 1 -- 172.21.15.46:0/164324103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0052e0 con 0x7eff440b1900 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff40ff9640 1 -- 172.21.15.46:0/164324103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3c005440 con 0x7eff440b1900 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff440b21e0 con 0x7eff440b1900 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff440ad820 con 0x7eff440b1900 2023-12-16T21:43:34.280 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff40ff9640 1 -- 172.21.15.46:0/164324103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3c003820 con 0x7eff440b1900 2023-12-16T21:43:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff440b3750 con 0x7eff440b1900 2023-12-16T21:43:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff40ff9640 1 -- 172.21.15.46:0/164324103 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7eff3c003980 con 0x7eff440b1900 2023-12-16T21:43:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff40ff9640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7eff300664a0 0x7eff30068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff40ff9640 1 -- 172.21.15.46:0/164324103 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff3c083800 con 0x7eff440b1900 2023-12-16T21:43:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.275+0000 7eff437fe640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7eff300664a0 0x7eff30068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.279+0000 7eff437fe640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7eff300664a0 0x7eff30068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7eff440b0dd0 tx=0x7eff3800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:34.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.279+0000 7eff40ff9640 1 -- 172.21.15.46:0/164324103 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7eff3c050c90 con 0x7eff440b1900 2023-12-16T21:43:34.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff440689d0 con 0x7eff440b1900 2023-12-16T21:43:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff40ff9640 1 -- 172.21.15.46:0/164324103 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff3c0542b0 con 0x7eff440b1900 2023-12-16T21:43:34.450 INFO:teuthology.orchestra.run.smithi046.stderr:listed 0 entries 2023-12-16T21:43:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7eff300664a0 msgr2=0x7eff30068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7eff300664a0 0x7eff30068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7eff440b0dd0 tx=0x7eff3800b040 comp rx=0 tx=0).stop 2023-12-16T21:43:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff440b1900 msgr2=0x7eff440afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff440b1900 0x7eff440afb90 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7eff3c00e580 tx=0x7eff3c00ea40 comp rx=0 tx=0).stop 2023-12-16T21:43:34.452 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 shutdown_connections 2023-12-16T21:43:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7eff300664a0 0x7eff30068960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7eff441dc7f0 0x7eff441460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7eff440b1ce0 0x7eff441dc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 --2- 172.21.15.46:0/164324103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7eff440b1900 0x7eff440afb90 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 >> 172.21.15.46:0/164324103 conn(0x7eff440b85e0 msgr2=0x7eff440b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 shutdown_connections 2023-12-16T21:43:34.453 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.447+0000 7eff4a6ed640 1 -- 172.21.15.46:0/164324103 wait complete. 2023-12-16T21:43:34.470 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi046.front.sepia.ceph.com... 2023-12-16T21:43:34.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:34.470 DEBUG:teuthology.orchestra.run.smithi046:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-16T21:43:34.483 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi046.stderr:ceph-fuse[46609]: fuse finished with error 0 and tester_r 0 2023-12-16T21:43:34.484 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:43:34.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:34.485 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-16T21:43:34.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:34.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f21315ab640 1 Processor -- start 2023-12-16T21:43:34.829 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f21315ab640 1 -- start start 2023-12-16T21:43:34.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f21315ab640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f212c1404c0 0x7f212c140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f21315ab640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f212c14edd0 0x7f212c1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f21315ab640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f212c0b0ec0 con 0x7f212c0b6840 2023-12-16T21:43:34.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f21315ab640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c0b7700 con 0x7f212c14edd0 2023-12-16T21:43:34.831 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f21315ab640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212c0b5210 con 0x7f212c1404c0 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f2130daa640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f212c14edd0 0x7f212c1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f2130daa640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f212c14edd0 0x7f212c1511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50248/0 (socket says 172.21.15.46:50248) 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f2130daa640 1 -- 172.21.15.46:0/803499378 learned_addr learned my addr 172.21.15.46:0/803499378 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f212b7fe640 1 --2- 172.21.15.46:0/803499378 >> v2:172.21.15.161:3300/0 conn(0x7f212c1404c0 0x7f212c140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f2130daa640 1 -- 172.21.15.46:0/803499378 >> v2:172.21.15.161:3300/0 conn(0x7f212c1404c0 msgr2=0x7f212c140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f2130daa640 1 --2- 172.21.15.46:0/803499378 >> v2:172.21.15.161:3300/0 conn(0x7f212c1404c0 0x7f212c140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f2130daa640 1 -- 172.21.15.46:0/803499378 >> v1:172.21.15.161:6789/0 conn(0x7f212c0b6840 legacy=0x7f212c0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f2130daa640 1 -- 172.21.15.46:0/803499378 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f212c0b54f0 con 0x7f212c14edd0 2023-12-16T21:43:34.832 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.827+0000 7f212b7fe640 1 --2- 172.21.15.46:0/803499378 >> v2:172.21.15.161:3300/0 conn(0x7f212c1404c0 0x7f212c140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:34.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f2130daa640 1 --2- 172.21.15.46:0/803499378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c14edd0 0x7f212c1511c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f212c0b50d0 tx=0x7f212401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca3d470dd15d5f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:34.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212affd640 1 -- 172.21.15.46:0/803499378 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21240a5700 con 0x7f212c14edd0 2023-12-16T21:43:34.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212affd640 1 -- 172.21.15.46:0/803499378 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21240a5860 con 0x7f212c14edd0 2023-12-16T21:43:34.838 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212affd640 1 -- 172.21.15.46:0/803499378 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21240a5bf0 con 0x7f212c14edd0 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- 172.21.15.46:0/803499378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c14edd0 msgr2=0x7f212c1511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 --2- 172.21.15.46:0/803499378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c14edd0 0x7f212c1511c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f212c0b50d0 tx=0x7f212401c990 comp rx=0 tx=0).stop 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- 172.21.15.46:0/803499378 shutdown_connections 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 --2- 172.21.15.46:0/803499378 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c14edd0 0x7f212c1511c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 --2- 172.21.15.46:0/803499378 >> v2:172.21.15.161:3300/0 conn(0x7f212c1404c0 0x7f212c140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- 172.21.15.46:0/803499378 >> 172.21.15.46:0/803499378 conn(0x7f212c06a580 msgr2=0x7f212c06a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- 172.21.15.46:0/803499378 shutdown_connections 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- 172.21.15.46:0/803499378 wait complete. 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 Processor -- start 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- start start 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c0b6840 0x7f212c13e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.839 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f212c1404c0 0x7f212c150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f212c14edd0 0x7f212c13ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c150820 con 0x7f212c0b6840 2023-12-16T21:43:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212c151180 con 0x7f212c14edd0 2023-12-16T21:43:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212c1501d0 con 0x7f212c1404c0 2023-12-16T21:43:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c0b6840 0x7f212c13e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.840 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c0b6840 0x7f212c13e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50254/0 (socket says 172.21.15.46:50254) 2023-12-16T21:43:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 -- 172.21.15.46:0/183696287 learned_addr learned my addr 172.21.15.46:0/183696287 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 -- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f212c1404c0 msgr2=0x7f212c150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f212c1404c0 0x7f212c150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 -- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f212c14edd0 msgr2=0x7f212c13ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f212c14edd0 0x7f212c13ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:34.841 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 -- 172.21.15.46:0/183696287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124009b30 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f212bfff640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c0b6840 0x7f212c13e0e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f211c00c4c0 tx=0x7f211c001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f2128ff9640 1 -- 172.21.15.46:0/183696287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c005480 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f2128ff9640 1 -- 172.21.15.46:0/183696287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c003600 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f212c13f100 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f2128ff9640 1 -- 172.21.15.46:0/183696287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c003990 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.831+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f212c150440 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.835+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f212c150960 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.835+0000 7f2128ff9640 1 -- 172.21.15.46:0/183696287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f211c003af0 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.835+0000 7f2128ff9640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2118066470 0x7f2118068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.835+0000 7f2128ff9640 1 -- 172.21.15.46:0/183696287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f211c083390 con 0x7f212c0b6840 2023-12-16T21:43:34.842 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.835+0000 7f212b7fe640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2118066470 0x7f2118068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.835+0000 7f212b7fe640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2118066470 0x7f2118068930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f212c1310e0 tx=0x7f2120008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:34.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:34.839+0000 7f2128ff9640 1 -- 172.21.15.46:0/183696287 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f211c0508a0 con 0x7f212c0b6840 2023-12-16T21:43:35.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.023+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f212c0695d0 con 0x7f212c0b6840 2023-12-16T21:43:35.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.023+0000 7f2128ff9640 1 -- 172.21.15.46:0/183696287 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3359 (secure 0 0 0) 0x7f211c017090 con 0x7f212c0b6840 2023-12-16T21:43:35.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:35.028 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3332084711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3332084711},{"type":"v1","addr":"172.21.15.161:6835","nonce":3332084711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:43:19.618214+0000","modified":"2023-12-16T21:43:21.693210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/3724256427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3724256427},{"type":"v1","addr":"172.21.15.46:6833","nonce":3724256427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-16T21:43:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-16T21:43:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2118066470 msgr2=0x7f2118068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2118066470 0x7f2118068930 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f212c1310e0 tx=0x7f2120008040 comp rx=0 tx=0).stop 2023-12-16T21:43:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c0b6840 msgr2=0x7f212c13e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c0b6840 0x7f212c13e0e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f211c00c4c0 tx=0x7f211c001e80 comp rx=0 tx=0).stop 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 shutdown_connections 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2118066470 0x7f2118068930 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f212c14edd0 0x7f212c13ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f212c1404c0 0x7f212c150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 --2- 172.21.15.46:0/183696287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f212c0b6840 0x7f212c13e0e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 >> 172.21.15.46:0/183696287 conn(0x7f212c06a580 msgr2=0x7f212c0b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 shutdown_connections 2023-12-16T21:43:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.027+0000 7f21315ab640 1 -- 172.21.15.46:0/183696287 wait complete. 2023-12-16T21:43:35.044 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.159+0000 7f390b7bb640 1 Processor -- start 2023-12-16T21:43:35.165 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.159+0000 7f390b7bb640 1 -- start start 2023-12-16T21:43:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f390b7bb640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 0x7f39040ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f390b7bb640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f39040af2e0 0x7f39040ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f390b7bb640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39040af770 con 0x7f39040b4430 2023-12-16T21:43:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f390b7bb640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39040afc20 con 0x7f39040af2e0 2023-12-16T21:43:35.167 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f390b7bb640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39040aeea0 con 0x7f390406c920 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f39040b4430 0x7f3904147080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56098/0 (socket says 172.21.15.46:56098) 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 -- 172.21.15.46:0/2322799082 learned_addr learned my addr 172.21.15.46:0/2322799082 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909557640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 0x7f39040ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3908d56640 1 --2- 172.21.15.46:0/2322799082 >> v2:172.21.15.161:3300/0 conn(0x7f39040af2e0 0x7f39040ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1377753341 0 0) 0x7f39040af770 con 0x7f39040b4430 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38f0003530 con 0x7f39040b4430 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 750907816 0 0) 0x7f38f0003530 con 0x7f39040b4430 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 msgr2=0x7f39040ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 0x7f39040ad0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 >> v2:172.21.15.161:3300/0 conn(0x7f39040af2e0 msgr2=0x7f39040ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 --2- 172.21.15.46:0/2322799082 >> v2:172.21.15.161:3300/0 conn(0x7f39040af2e0 0x7f39040ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f390406cd00 con 0x7f39040b4430 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f38fc001680 con 0x7f39040b4430 2023-12-16T21:43:35.168 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 >> v1:172.21.15.46:6789/0 conn(0x7f39040b4430 legacy=0x7f3904147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f0003980 0x7f38f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 0x7f38f00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f0004d10 0x7f38f0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39040afc20 con 0x7f38f0004d10 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39040aeea0 con 0x7f38f0003980 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f000dc20 con 0x7f390406c920 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f0004d10 0x7f38f0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909557640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f0003980 0x7f38f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3908d56640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 0x7f38f00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 -- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 msgr2=0x7f38f00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f390406c920 0x7f38f00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 -- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f0003980 msgr2=0x7f38f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f38f0003980 0x7f38f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.169 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.163+0000 7f3909d58640 1 -- 172.21.15.46:0/2322799082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0003530 con 0x7f38f0004d10 2023-12-16T21:43:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f0004d10 0x7f38f0005190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c240 tx=0x7f38fc01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1fcdacd3711e033 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0058e0 con 0x7f38f0004d10 2023-12-16T21:43:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc005a40 con 0x7f38f0004d10 2023-12-16T21:43:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f38fbfff640 1 -- 172.21.15.46:0/2322799082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc005e30 con 0x7f38f0004d10 2023-12-16T21:43:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- 172.21.15.46:0/2322799082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f0004d10 msgr2=0x7f38f0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 --2- 172.21.15.46:0/2322799082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f38f0004d10 0x7f38f0005190 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00c240 tx=0x7f38fc01cc60 comp rx=0 tx=0).stop 2023-12-16T21:43:35.171 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 -- 172.21.15.46:0/2322799082 reap_dead start 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- 172.21.15.46:0/2322799082 shutdown_connections 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- 172.21.15.46:0/2322799082 >> 172.21.15.46:0/2322799082 conn(0x7f39040b48e0 msgr2=0x7f39040b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- 172.21.15.46:0/2322799082 shutdown_connections 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- 172.21.15.46:0/2322799082 wait complete. 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 Processor -- start 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- start start 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3904070010 0x7f39041e2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39040703f0 0x7f39041e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39041e2de0 0x7f39041d1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3904121050 con 0x7f39041e2de0 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39041ded90 con 0x7f3904070010 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39041df050 con 0x7f39040703f0 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39041e2de0 0x7f39041d1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909557640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3904070010 0x7f39041e2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909557640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3904070010 0x7f39041e2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:51524/0 (socket says 172.21.15.46:51524) 2023-12-16T21:43:35.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909557640 1 -- 172.21.15.46:0/3182831560 learned_addr learned my addr 172.21.15.46:0/3182831560 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3908d56640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39040703f0 0x7f39041e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 -- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39040703f0 msgr2=0x7f39041e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39040703f0 0x7f39041e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 -- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3904070010 msgr2=0x7f39041e2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3904070010 0x7f39041e2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 -- 172.21.15.46:0/3182831560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39041df330 con 0x7f39041e2de0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f3909d58640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39041e2de0 0x7f39041d1920 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009000 tx=0x7f38f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f38f9ffb640 1 -- 172.21.15.46:0/3182831560 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc001640 con 0x7f39041e2de0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f38f9ffb640 1 -- 172.21.15.46:0/3182831560 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc001c60 con 0x7f39041e2de0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39041df610 con 0x7f39041e2de0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f38f9ffb640 1 -- 172.21.15.46:0/3182831560 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc004ab0 con 0x7f39041e2de0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.167+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390411fa30 con 0x7f39041e2de0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.171+0000 7f38f9ffb640 1 -- 172.21.15.46:0/3182831560 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f38fc004c10 con 0x7f39041e2de0 2023-12-16T21:43:35.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.171+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39040679e0 con 0x7f39041e2de0 2023-12-16T21:43:35.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.171+0000 7f38f9ffb640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f38e80664a0 0x7f38e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.171+0000 7f38f9ffb640 1 -- 172.21.15.46:0/3182831560 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38fc11ce20 con 0x7f39041e2de0 2023-12-16T21:43:35.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.171+0000 7f3909557640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f38e80664a0 0x7f38e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.171+0000 7f3909557640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f38e80664a0 0x7f38e8068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f39000075c0 tx=0x7f390000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.178 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.175+0000 7f38f9ffb640 1 -- 172.21.15.46:0/3182831560 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f38fc0ea350 con 0x7f39041e2de0 2023-12-16T21:43:35.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.331+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39040688f0 con 0x7f39041e2de0 2023-12-16T21:43:35.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f38f9ffb640 1 -- 172.21.15.46:0/3182831560 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f38fc0ed970 con 0x7f39041e2de0 2023-12-16T21:43:35.337 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:35.338 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:21.267009+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:43:17.887506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-16T21:43:18.612695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f38e80664a0 msgr2=0x7f38e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f38e80664a0 0x7f38e8068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f39000075c0 tx=0x7f390000b040 comp rx=0 tx=0).stop 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39041e2de0 msgr2=0x7f39041d1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39041e2de0 0x7f39041d1920 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f38fc009000 tx=0x7f38f0005800 comp rx=0 tx=0).stop 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 shutdown_connections 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f38e80664a0 0x7f38e8068960 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39041e2de0 0x7f39041d1920 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f39040703f0 0x7f39041e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 --2- 172.21.15.46:0/3182831560 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f3904070010 0x7f39041e2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 >> 172.21.15.46:0/3182831560 conn(0x7f39040b48e0 msgr2=0x7f39040b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:35.341 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 shutdown_connections 2023-12-16T21:43:35.342 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.335+0000 7f390b7bb640 1 -- 172.21.15.46:0/3182831560 wait complete. 2023-12-16T21:43:35.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-16T21:42:28.427811+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-16T21:43:17.887506+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-16T21:43:18.612695+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-16T21:43:35.358 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-16T21:43:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.483+0000 7f4c4f8cd640 1 Processor -- start 2023-12-16T21:43:35.488 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.483+0000 7f4c4f8cd640 1 -- start start 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4c480b3ff0 0x7f4c48147c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c481481a0 0x7f4c4814ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480b1f80 con 0x7f4c480b5f80 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c48148630 con 0x7f4c481481a0 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c480b6360 con 0x7f4c480b3ff0 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c481481a0 0x7f4c4814ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4c481481a0 0x7f4c4814ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50286/0 (socket says 172.21.15.46:50286) 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 -- 172.21.15.46:0/2903856963 learned_addr learned my addr 172.21.15.46:0/2903856963 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4ce68640 1 --2- 172.21.15.46:0/2903856963 >> v2:172.21.15.161:3300/0 conn(0x7f4c480b3ff0 0x7f4c48147c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 -- 172.21.15.46:0/2903856963 >> v2:172.21.15.161:3300/0 conn(0x7f4c480b3ff0 msgr2=0x7f4c48147c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 --2- 172.21.15.46:0/2903856963 >> v2:172.21.15.161:3300/0 conn(0x7f4c480b3ff0 0x7f4c48147c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 -- 172.21.15.46:0/2903856963 >> v1:172.21.15.161:6789/0 conn(0x7f4c480b5f80 legacy=0x7f4c480b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:35.490 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 -- 172.21.15.46:0/2903856963 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480b4760 con 0x7f4c481481a0 2023-12-16T21:43:35.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4de6a640 1 --2- 172.21.15.46:0/2903856963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c481481a0 0x7f4c4814ee90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4c480bae80 tx=0x7f4c4001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d3cbc8988246aa server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.491 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c3ffff640 1 -- 172.21.15.46:0/2903856963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400a5740 con 0x7f4c481481a0 2023-12-16T21:43:35.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c3ffff640 1 -- 172.21.15.46:0/2903856963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c400a58a0 con 0x7f4c481481a0 2023-12-16T21:43:35.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c3ffff640 1 -- 172.21.15.46:0/2903856963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400a5c30 con 0x7f4c481481a0 2023-12-16T21:43:35.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/2903856963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c481481a0 msgr2=0x7f4c4814ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/2903856963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c481481a0 0x7f4c4814ee90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4c480bae80 tx=0x7f4c4001c990 comp rx=0 tx=0).stop 2023-12-16T21:43:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/2903856963 shutdown_connections 2023-12-16T21:43:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/2903856963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c481481a0 0x7f4c4814ee90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/2903856963 >> v2:172.21.15.161:3300/0 conn(0x7f4c480b3ff0 0x7f4c48147c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/2903856963 >> 172.21.15.46:0/2903856963 conn(0x7f4c4806a980 msgr2=0x7f4c4806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/2903856963 shutdown_connections 2023-12-16T21:43:35.493 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.487+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/2903856963 wait complete. 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 Processor -- start 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 -- start start 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c480b3ff0 0x7f4c48146f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c480b5f80 0x7f4c481e2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c481481a0 0x7f4c481e2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48145c70 con 0x7f4c480b3ff0 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48145f50 con 0x7f4c480b5f80 2023-12-16T21:43:35.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48146230 con 0x7f4c481481a0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c480b3ff0 0x7f4c48146f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c480b3ff0 0x7f4c48146f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50292/0 (socket says 172.21.15.46:50292) 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 -- 172.21.15.46:0/99257828 learned_addr learned my addr 172.21.15.46:0/99257828 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4ce68640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c480b5f80 0x7f4c481e2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4de6a640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c481481a0 0x7f4c481e2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 -- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c481481a0 msgr2=0x7f4c481e2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c481481a0 0x7f4c481e2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 -- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c480b5f80 msgr2=0x7f4c481e2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c480b5f80 0x7f4c481e2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 -- 172.21.15.46:0/99257828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c40009b30 con 0x7f4c480b3ff0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4d669640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c480b3ff0 0x7f4c48146f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c450 tx=0x7f4c4400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c3dffb640 1 -- 172.21.15.46:0/99257828 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44012830 con 0x7f4c480b3ff0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c3dffb640 1 -- 172.21.15.46:0/99257828 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c440129b0 con 0x7f4c480b3ff0 2023-12-16T21:43:35.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4814e140 con 0x7f4c480b3ff0 2023-12-16T21:43:35.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c3dffb640 1 -- 172.21.15.46:0/99257828 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c44012d60 con 0x7f4c480b3ff0 2023-12-16T21:43:35.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4806d4a0 con 0x7f4c480b3ff0 2023-12-16T21:43:35.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c480b1790 con 0x7f4c480b3ff0 2023-12-16T21:43:35.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.491+0000 7f4c3dffb640 1 -- 172.21.15.46:0/99257828 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f4c4401c460 con 0x7f4c480b3ff0 2023-12-16T21:43:35.497 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.495+0000 7f4c3dffb640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4c2c066470 0x7f4c2c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.495+0000 7f4c3dffb640 1 -- 172.21.15.46:0/99257828 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c44083ca0 con 0x7f4c480b3ff0 2023-12-16T21:43:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.495+0000 7f4c4ce68640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4c2c066470 0x7f4c2c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.495+0000 7f4c4ce68640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4c2c066470 0x7f4c2c068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c4814ebc0 tx=0x7f4c38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.499+0000 7f4c3dffb640 1 -- 172.21.15.46:0/99257828 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f4c44051120 con 0x7f4c480b3ff0 2023-12-16T21:43:35.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.651+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4c48145db0 con 0x7f4c480b3ff0 2023-12-16T21:43:35.657 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c3dffb640 1 -- 172.21.15.46:0/99257828 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4c48145db0 con 0x7f4c480b3ff0 2023-12-16T21:43:35.657 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4c2c066470 msgr2=0x7f4c2c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4c2c066470 0x7f4c2c068930 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c4814ebc0 tx=0x7f4c38008040 comp rx=0 tx=0).stop 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c480b3ff0 msgr2=0x7f4c48146f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c480b3ff0 0x7f4c48146f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400c450 tx=0x7f4c4400c910 comp rx=0 tx=0).stop 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 shutdown_connections 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4c2c066470 0x7f4c2c068930 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4c481481a0 0x7f4c481e2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4c480b5f80 0x7f4c481e2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 --2- 172.21.15.46:0/99257828 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c480b3ff0 0x7f4c48146f20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 >> 172.21.15.46:0/99257828 conn(0x7f4c4806a980 msgr2=0x7f4c480b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 shutdown_connections 2023-12-16T21:43:35.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.655+0000 7f4c4f8cd640 1 -- 172.21.15.46:0/99257828 wait complete. 2023-12-16T21:43:35.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-16T21:43:35.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.799+0000 7fa6413c4640 1 Processor -- start 2023-12-16T21:43:35.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.799+0000 7fa6413c4640 1 -- start start 2023-12-16T21:43:35.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa6413c4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0b1eb0 0x7fa63c0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa6413c4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa63c0af300 0x7fa63c144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa6413c4640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63c0b1730 con 0x7fa63c0b0470 2023-12-16T21:43:35.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa6413c4640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c06cd70 con 0x7fa63c0af300 2023-12-16T21:43:35.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa6413c4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0b0ad0 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa63c0af300 0x7fa63c144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63affd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0b1eb0 0x7fa63c0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fa63c0af300 0x7fa63c144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:51556/0 (socket says 172.21.15.46:51556) 2023-12-16T21:43:35.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63affd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0b1eb0 0x7fa63c0aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:52828/0 (socket says 172.21.15.46:52828) 2023-12-16T21:43:35.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 -- 172.21.15.46:0/2143906368 learned_addr learned my addr 172.21.15.46:0/2143906368 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:35.806 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 -- 172.21.15.46:0/2143906368 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0b1eb0 msgr2=0x7fa63c0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:35.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 --2- 172.21.15.46:0/2143906368 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0b1eb0 0x7fa63c0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 -- 172.21.15.46:0/2143906368 >> v1:172.21.15.161:6789/0 conn(0x7fa63c0b0470 legacy=0x7fa63c1479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.807 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 -- 172.21.15.46:0/2143906368 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c06c7f0 con 0x7fa63c0af300 2023-12-16T21:43:35.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.803+0000 7fa63a7fc640 1 --2- 172.21.15.46:0/2143906368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0af300 0x7fa63c144ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa624009e50 tx=0x7fa62401c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf80ba6b5a9e4ae9 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa639ffb640 1 -- 172.21.15.46:0/2143906368 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a57b0 con 0x7fa63c0af300 2023-12-16T21:43:35.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa639ffb640 1 -- 172.21.15.46:0/2143906368 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6240a5910 con 0x7fa63c0af300 2023-12-16T21:43:35.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa639ffb640 1 -- 172.21.15.46:0/2143906368 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240a5ca0 con 0x7fa63c0af300 2023-12-16T21:43:35.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- 172.21.15.46:0/2143906368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0af300 msgr2=0x7fa63c144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.809 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 --2- 172.21.15.46:0/2143906368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0af300 0x7fa63c144ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa624009e50 tx=0x7fa62401c8f0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- 172.21.15.46:0/2143906368 shutdown_connections 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 --2- 172.21.15.46:0/2143906368 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0af300 0x7fa63c144ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 --2- 172.21.15.46:0/2143906368 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0b1eb0 0x7fa63c0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- 172.21.15.46:0/2143906368 >> 172.21.15.46:0/2143906368 conn(0x7fa63c0b4900 msgr2=0x7fa63c0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- 172.21.15.46:0/2143906368 shutdown_connections 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- 172.21.15.46:0/2143906368 wait complete. 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 Processor -- start 2023-12-16T21:43:35.810 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- start start 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0af300 0x7fa63c146ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0b0470 0x7fa63c1dfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c0b1eb0 0x7fa63c1e2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0715d0 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c13f270 con 0x7fa63c0b0470 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c1df6e0 con 0x7fa63c0af300 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c0b1eb0 0x7fa63c1e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c0b1eb0 0x7fa63c1e2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50296/0 (socket says 172.21.15.46:50296) 2023-12-16T21:43:35.813 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 -- 172.21.15.46:0/3611654127 learned_addr learned my addr 172.21.15.46:0/3611654127 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63a7fc640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0b0470 0x7fa63c1dfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63affd640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0af300 0x7fa63c146ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 -- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0af300 msgr2=0x7fa63c146ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0af300 0x7fa63c146ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 -- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0b0470 msgr2=0x7fa63c1dfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0b0470 0x7fa63c1dfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 -- 172.21.15.46:0/3611654127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa624009b30 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa63b7fe640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c0b1eb0 0x7fa63c1e2900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c3c0 tx=0x7fa63400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6337fe640 1 -- 172.21.15.46:0/3611654127 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6340052d0 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6337fe640 1 -- 172.21.15.46:0/3611654127 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa634005430 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c1477e0 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6337fe640 1 -- 172.21.15.46:0/3611654127 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634007260 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.814 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.807+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c06e780 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.811+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c13f490 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.811+0000 7fa6337fe640 1 -- 172.21.15.46:0/3611654127 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fa6340073c0 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.815 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.811+0000 7fa6337fe640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa61c0664a0 0x7fa61c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:35.816 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.811+0000 7fa6337fe640 1 -- 172.21.15.46:0/3611654127 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa634083010 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.811+0000 7fa63affd640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa61c0664a0 0x7fa61c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:35.817 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.811+0000 7fa63affd640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa61c0664a0 0x7fa61c068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009a50 tx=0x7fa62c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:35.821 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.815+0000 7fa6337fe640 1 -- 172.21.15.46:0/3611654127 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fa6340b4290 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa63c1df900 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.970 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6337fe640 1 -- 172.21.15.46:0/3611654127 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa634050520 con 0x7fa63c0b1eb0 2023-12-16T21:43:35.970 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa61c0664a0 msgr2=0x7fa61c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa61c0664a0 0x7fa61c068960 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa62c009a50 tx=0x7fa62c008040 comp rx=0 tx=0).stop 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c0b1eb0 msgr2=0x7fa63c1e2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c0b1eb0 0x7fa63c1e2900 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c3c0 tx=0x7fa63400c880 comp rx=0 tx=0).stop 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 shutdown_connections 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fa61c0664a0 0x7fa61c068960 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa63c0b1eb0 0x7fa63c1e2900 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fa63c0b0470 0x7fa63c1dfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 --2- 172.21.15.46:0/3611654127 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fa63c0af300 0x7fa63c146ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 >> 172.21.15.46:0/3611654127 conn(0x7fa63c0b4900 msgr2=0x7fa63c0b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:35.972 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 shutdown_connections 2023-12-16T21:43:35.973 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:35.967+0000 7fa6413c4640 1 -- 172.21.15.46:0/3611654127 wait complete. 2023-12-16T21:43:35.988 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-16T21:43:36.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 Processor -- start 2023-12-16T21:43:36.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 -- start start 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12fc0b1eb0 0x7f12fc0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f12fc0af300 0x7f12fc144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0b0470 0x7f12fc144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc06cd70 con 0x7f12fc0b1eb0 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12fc0b0ad0 con 0x7f12fc0af300 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f13030f3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0b1730 con 0x7f12fc0b0470 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12fc0b1eb0 0x7f12fc0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f12fc0b1eb0 0x7f12fc0aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50312/0 (socket says 172.21.15.46:50312) 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 -- 172.21.15.46:0/3160232548 learned_addr learned my addr 172.21.15.46:0/3160232548 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f12fbfff640 1 --2- 172.21.15.46:0/3160232548 >> v2:172.21.15.161:3300/0 conn(0x7f12fc0af300 0x7f12fc144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1301690640 1 --2- 172.21.15.46:0/3160232548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0b0470 0x7f12fc144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 -- 172.21.15.46:0/3160232548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0b0470 msgr2=0x7f12fc144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 --2- 172.21.15.46:0/3160232548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0b0470 0x7f12fc144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 -- 172.21.15.46:0/3160232548 >> v2:172.21.15.161:3300/0 conn(0x7f12fc0af300 msgr2=0x7f12fc144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 --2- 172.21.15.46:0/3160232548 >> v2:172.21.15.161:3300/0 conn(0x7f12fc0af300 0x7f12fc144ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1300e8f640 1 -- 172.21.15.46:0/3160232548 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12fc06c7f0 con 0x7f12fc0b1eb0 2023-12-16T21:43:36.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.111+0000 7f1301690640 1 --2- 172.21.15.46:0/3160232548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0b0470 0x7f12fc144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:36.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f1300e8f640 1 --2- 172.21.15.46:0/3160232548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b1eb0 0x7f12fc0aff30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0b59c0 tx=0x7f12ec01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d69306054456f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fb7fe640 1 -- 172.21.15.46:0/3160232548 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0ac600 con 0x7f12fc0b1eb0 2023-12-16T21:43:36.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fb7fe640 1 -- 172.21.15.46:0/3160232548 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec0a8050 con 0x7f12fc0b1eb0 2023-12-16T21:43:36.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fb7fe640 1 -- 172.21.15.46:0/3160232548 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec0ac980 con 0x7f12fc0b1eb0 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- 172.21.15.46:0/3160232548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b1eb0 msgr2=0x7f12fc0aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 --2- 172.21.15.46:0/3160232548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b1eb0 0x7f12fc0aff30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f12fc0b59c0 tx=0x7f12ec01c3e0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- 172.21.15.46:0/3160232548 shutdown_connections 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 --2- 172.21.15.46:0/3160232548 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0b0470 0x7f12fc144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 --2- 172.21.15.46:0/3160232548 >> v2:172.21.15.161:3300/0 conn(0x7f12fc0af300 0x7f12fc144ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 --2- 172.21.15.46:0/3160232548 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b1eb0 0x7f12fc0aff30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- 172.21.15.46:0/3160232548 >> 172.21.15.46:0/3160232548 conn(0x7f12fc0b4900 msgr2=0x7f12fc0b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- 172.21.15.46:0/3160232548 shutdown_connections 2023-12-16T21:43:36.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- 172.21.15.46:0/3160232548 wait complete. 2023-12-16T21:43:36.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 Processor -- start 2023-12-16T21:43:36.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- start start 2023-12-16T21:43:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0af300 0x7f12fc1de820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b0470 0x7f12fc0ad890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc0b1eb0 0x7f12fc0addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0ace20 con 0x7f12fc0b0470 2023-12-16T21:43:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0714c0 con 0x7f12fc0b1eb0 2023-12-16T21:43:36.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f13030f3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12fc0ad090 con 0x7f12fc0af300 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b0470 0x7f12fc0ad890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b0470 0x7f12fc0ad890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50314/0 (socket says 172.21.15.46:50314) 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 -- 172.21.15.46:0/3431524753 learned_addr learned my addr 172.21.15.46:0/3431524753 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f1300e8f640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0af300 0x7f12fc1de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f1301690640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc0b1eb0 0x7f12fc0addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 -- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0af300 msgr2=0x7f12fc1de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0af300 0x7f12fc1de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 -- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc0b1eb0 msgr2=0x7f12fc0addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.123 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc0b1eb0 0x7f12fc0addd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 -- 172.21.15.46:0/3431524753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0094d0 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f1300e8f640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0af300 0x7f12fc1de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12fbfff640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b0470 0x7f12fc0ad890 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f12e400d800 tx=0x7f12e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12f97fa640 1 -- 172.21.15.46:0/3431524753 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400b6c0 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12f97fa640 1 -- 172.21.15.46:0/3431524753 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12e400b820 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.115+0000 7f12f97fa640 1 -- 172.21.15.46:0/3431524753 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12e400bb90 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12fc0ad370 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12fc1df2e0 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12fc068960 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f12f97fa640 1 -- 172.21.15.46:0/3431524753 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f12e400bcf0 con 0x7f12fc0b0470 2023-12-16T21:43:36.124 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f12f97fa640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f12dc066470 0x7f12dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f12f97fa640 1 -- 172.21.15.46:0/3431524753 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12e40843f0 con 0x7f12fc0b0470 2023-12-16T21:43:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f1300e8f640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f12dc066470 0x7f12dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.125 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.119+0000 7f1300e8f640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f12dc066470 0x7f12dc068930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0ae000 tx=0x7f12ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.126 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.123+0000 7f12f97fa640 1 -- 172.21.15.46:0/3431524753 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f12e4051880 con 0x7f12fc0b0470 2023-12-16T21:43:36.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f12fc073e80 con 0x7f12fc0b0470 2023-12-16T21:43:36.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f12f97fa640 1 -- 172.21.15.46:0/3431524753 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f12e4054ea0 con 0x7f12fc0b0470 2023-12-16T21:43:36.282 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2023-12-16T21:43:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f12dc066470 msgr2=0x7f12dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f12dc066470 0x7f12dc068930 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f12ec0ae000 tx=0x7f12ec008040 comp rx=0 tx=0).stop 2023-12-16T21:43:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b0470 msgr2=0x7f12fc0ad890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b0470 0x7f12fc0ad890 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f12e400d800 tx=0x7f12e400dcc0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 shutdown_connections 2023-12-16T21:43:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f12dc066470 0x7f12dc068930 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f12fc0b1eb0 0x7f12fc0addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12fc0b0470 0x7f12fc0ad890 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 --2- 172.21.15.46:0/3431524753 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f12fc0af300 0x7f12fc1de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 >> 172.21.15.46:0/3431524753 conn(0x7f12fc0b4900 msgr2=0x7f12fc143060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 shutdown_connections 2023-12-16T21:43:36.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.279+0000 7f13030f3640 1 -- 172.21.15.46:0/3431524753 wait complete. 2023-12-16T21:43:36.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.423+0000 7ff39d2c8640 1 Processor -- start 2023-12-16T21:43:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.423+0000 7ff39d2c8640 1 -- start start 2023-12-16T21:43:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff39d2c8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3980af2e0 0x7ff398147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff39d2c8640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff3980ad7c0 0x7ff3981475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff39d2c8640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3980af770 con 0x7ff39806c920 2023-12-16T21:43:36.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff39d2c8640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3980afc20 con 0x7ff3980ad7c0 2023-12-16T21:43:36.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff39d2c8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980aeea0 con 0x7ff3980af2e0 2023-12-16T21:43:36.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff396ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff39806c920 0x7ff3980ad0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56112/0 (socket says 172.21.15.46:56112) 2023-12-16T21:43:36.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff396ffd640 1 -- 172.21.15.46:0/829089929 learned_addr learned my addr 172.21.15.46:0/829089929 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3967fc640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3980af2e0 0x7ff398147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3977fe640 1 --2- 172.21.15.46:0/829089929 >> v2:172.21.15.161:3300/0 conn(0x7ff3980ad7c0 0x7ff3981475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010884675 0 0) 0x7ff3980af770 con 0x7ff39806c920 2023-12-16T21:43:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff380003530 con 0x7ff39806c920 2023-12-16T21:43:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641879396 0 0) 0x7ff380003530 con 0x7ff39806c920 2023-12-16T21:43:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3980af2e0 msgr2=0x7ff398147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3980af2e0 0x7ff398147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 >> v2:172.21.15.161:3300/0 conn(0x7ff3980ad7c0 msgr2=0x7ff3981475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 --2- 172.21.15.46:0/829089929 >> v2:172.21.15.161:3300/0 conn(0x7ff3980ad7c0 0x7ff3981475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff398147b00 con 0x7ff39806c920 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7ff390001a00 con 0x7ff39806c920 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 >> v1:172.21.15.46:6789/0 conn(0x7ff39806c920 legacy=0x7ff3980ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff380003980 0x7ff380003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3800047d0 0x7ff380004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff380005190 0x7ff38000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980afc20 con 0x7ff3800047d0 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980aeea0 con 0x7ff380003980 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38000bed0 con 0x7ff380005190 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3967fc640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3800047d0 0x7ff380004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff396ffd640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff380003980 0x7ff380003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3977fe640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff380005190 0x7ff38000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3967fc640 1 -- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff380005190 msgr2=0x7ff38000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3967fc640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff380005190 0x7ff38000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3967fc640 1 -- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff380003980 msgr2=0x7ff380003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3967fc640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff380003980 0x7ff380003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3967fc640 1 -- 172.21.15.46:0/829089929 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380003530 con 0x7ff3800047d0 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff3977fe640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff380005190 0x7ff38000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.427+0000 7ff396ffd640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff380003980 0x7ff380003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3967fc640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3800047d0 0x7ff380004c50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009ae0 tx=0x7ff38c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae1f02203338c4d server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0a5620 con 0x7ff3800047d0 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c0a5780 con 0x7ff3800047d0 2023-12-16T21:43:36.441 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff395ffb640 1 -- 172.21.15.46:0/829089929 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c0a5b30 con 0x7ff3800047d0 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- 172.21.15.46:0/829089929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3800047d0 msgr2=0x7ff380004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 --2- 172.21.15.46:0/829089929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3800047d0 0x7ff380004c50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009ae0 tx=0x7ff38c01ca50 comp rx=0 tx=0).stop 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 -- 172.21.15.46:0/829089929 reap_dead start 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- 172.21.15.46:0/829089929 shutdown_connections 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- 172.21.15.46:0/829089929 >> 172.21.15.46:0/829089929 conn(0x7ff3980b48e0 msgr2=0x7ff3980b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- 172.21.15.46:0/829089929 shutdown_connections 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- 172.21.15.46:0/829089929 wait complete. 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 Processor -- start 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- start start 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3800047d0 0x7ff3980b6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39813b190 0x7ff3981dee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3981df3a0 0x7ff398138f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980b5f60 con 0x7ff3981df3a0 2023-12-16T21:43:36.442 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3980b71e0 con 0x7ff39813b190 2023-12-16T21:43:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39813be50 con 0x7ff3800047d0 2023-12-16T21:43:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3981df3a0 0x7ff398138f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3981df3a0 0x7ff398138f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50332/0 (socket says 172.21.15.46:50332) 2023-12-16T21:43:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 -- 172.21.15.46:0/3957765332 learned_addr learned my addr 172.21.15.46:0/3957765332 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff396ffd640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3800047d0 0x7ff3980b6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3967fc640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39813b190 0x7ff3981dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 -- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3800047d0 msgr2=0x7ff3980b6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3800047d0 0x7ff3980b6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 -- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39813b190 msgr2=0x7ff3981dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39813b190 0x7ff3981dee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 -- 172.21.15.46:0/3957765332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3980b73f0 con 0x7ff3981df3a0 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3967fc640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39813b190 0x7ff3981dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff3977fe640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3981df3a0 0x7ff398138f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff3980adc20 tx=0x7ff388010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff37f7fe640 1 -- 172.21.15.46:0/3957765332 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800e8a0 con 0x7ff3981df3a0 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff37f7fe640 1 -- 172.21.15.46:0/3957765332 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38800ea00 con 0x7ff3981df3a0 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39813a370 con 0x7ff3981df3a0 2023-12-16T21:43:36.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff37f7fe640 1 -- 172.21.15.46:0/3957765332 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800edb0 con 0x7ff3981df3a0 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.431+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39813b930 con 0x7ff3981df3a0 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.435+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3980679e0 con 0x7ff3981df3a0 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.435+0000 7ff37f7fe640 1 -- 172.21.15.46:0/3957765332 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7ff38801c400 con 0x7ff3981df3a0 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.435+0000 7ff37f7fe640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff384066470 0x7ff384068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.435+0000 7ff37f7fe640 1 -- 172.21.15.46:0/3957765332 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff388083c60 con 0x7ff3981df3a0 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.435+0000 7ff396ffd640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff384066470 0x7ff384068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.435+0000 7ff396ffd640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff384066470 0x7ff384068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff3900014b0 tx=0x7ff390009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.445 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.439+0000 7ff37f7fe640 1 -- 172.21.15.46:0/3957765332 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7ff388051220 con 0x7ff3981df3a0 2023-12-16T21:43:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.631+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff39813bb20 con 0x7ff3981df3a0 2023-12-16T21:43:36.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.631+0000 7ff37f7fe640 1 -- 172.21.15.46:0/3957765332 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3359 (secure 0 0 0) 0x7ff388054840 con 0x7ff3981df3a0 2023-12-16T21:43:36.638 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:36.638 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3332084711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3332084711},{"type":"v1","addr":"172.21.15.161:6835","nonce":3332084711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:43:19.618214+0000","modified":"2023-12-16T21:43:21.693210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/3724256427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3724256427},{"type":"v1","addr":"172.21.15.46:6833","nonce":3724256427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-16T21:43:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-16T21:43:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff384066470 msgr2=0x7ff384068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff384066470 0x7ff384068930 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff3900014b0 tx=0x7ff390009040 comp rx=0 tx=0).stop 2023-12-16T21:43:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3981df3a0 msgr2=0x7ff398138f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3981df3a0 0x7ff398138f20 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff3980adc20 tx=0x7ff388010c20 comp rx=0 tx=0).stop 2023-12-16T21:43:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 shutdown_connections 2023-12-16T21:43:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff384066470 0x7ff384068930 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff3981df3a0 0x7ff398138f20 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff39813b190 0x7ff3981dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 --2- 172.21.15.46:0/3957765332 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff3800047d0 0x7ff3980b6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 >> 172.21.15.46:0/3957765332 conn(0x7ff3980b48e0 msgr2=0x7ff3980b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 shutdown_connections 2023-12-16T21:43:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.635+0000 7ff39d2c8640 1 -- 172.21.15.46:0/3957765332 wait complete. 2023-12-16T21:43:36.655 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:36.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:43:36.736 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:43:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.771+0000 7fc5b0c73640 1 Processor -- start 2023-12-16T21:43:36.776 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.771+0000 7fc5b0c73640 1 -- start start 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5b0c73640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 0x7fc5ac0b4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5b0c73640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac147bb0 con 0x7fc5ac0b3ef0 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5b0c73640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ac0b1e30 con 0x7fc5ac06caa0 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5b0c73640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0af2e0 con 0x7fc5ac06c6c0 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc5ac0b3ef0 0x7fc5ac0b0270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56122/0 (socket says 172.21.15.46:56122) 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 -- 172.21.15.46:0/672959472 learned_addr learned my addr 172.21.15.46:0/672959472 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5aa59c640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 0x7fc5ac0b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822757995 0 0) 0x7fc5ac147bb0 con 0x7fc5ac0b3ef0 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc594003530 con 0x7fc5ac0b3ef0 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 129970122 0 0) 0x7fc594003530 con 0x7fc5ac0b3ef0 2023-12-16T21:43:36.779 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 msgr2=0x7fc5ac0b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 0x7fc5ac0b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 >> v1:172.21.15.161:6789/0 conn(0x7fc5ac06caa0 legacy=0x7fc5ac1483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac0af700 con 0x7fc5ac0b3ef0 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fc59c001680 con 0x7fc5ac0b3ef0 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5aa59c640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 0x7fc5ac0b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 >> v1:172.21.15.46:6789/0 conn(0x7fc5ac0b3ef0 legacy=0x7fc5ac0b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 0x7fc5940039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594004d10 0x7fc594005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac0af2e0 con 0x7fc594003ef0 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59400b9f0 con 0x7fc594004d10 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59400bbb0 con 0x7fc5ac06c6c0 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5aa59c640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 0x7fc5940039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5aad9d640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594004d10 0x7fc594005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 -- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 msgr2=0x7fc5940039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac06c6c0 0x7fc5940039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 -- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594004d10 msgr2=0x7fc594005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc594004d10 0x7fc594005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.775+0000 7fc5a9d9b640 1 -- 172.21.15.46:0/672959472 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a0009a70 con 0x7fc594003ef0 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5940047d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00c210 tx=0x7fc59c01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167bda04e05ded29 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c005ae0 con 0x7fc594003ef0 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc59c005c60 con 0x7fc594003ef0 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5a959a640 1 -- 172.21.15.46:0/672959472 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59c0072b0 con 0x7fc594003ef0 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- 172.21.15.46:0/672959472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 msgr2=0x7fc5940047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 --2- 172.21.15.46:0/672959472 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5940047d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc59c00c210 tx=0x7fc59c01cb60 comp rx=0 tx=0).stop 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aad9d640 1 -- 172.21.15.46:0/672959472 reap_dead start 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- 172.21.15.46:0/672959472 shutdown_connections 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- 172.21.15.46:0/672959472 >> 172.21.15.46:0/672959472 conn(0x7fc5ac0b48e0 msgr2=0x7fc5ac0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- 172.21.15.46:0/672959472 shutdown_connections 2023-12-16T21:43:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- 172.21.15.46:0/672959472 wait complete. 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 Processor -- start 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- start start 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5ac1478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5ac144bc0 0x7fc5ac1e0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac1e1340 0x7fc5ac136ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac144180 con 0x7fc594003ef0 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac1e17f0 con 0x7fc5ac144bc0 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ac1450a0 con 0x7fc5ac1e1340 2023-12-16T21:43:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5ac1478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5ac1478b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50344/0 (socket says 172.21.15.46:50344) 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 -- 172.21.15.46:0/2854114705 learned_addr learned my addr 172.21.15.46:0/2854114705 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aad9d640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac1e1340 0x7fc5ac136ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5ac144bc0 0x7fc5ac1e0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 -- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac1e1340 msgr2=0x7fc5ac136ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac1e1340 0x7fc5ac136ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 -- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5ac144bc0 msgr2=0x7fc5ac1e0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5ac144bc0 0x7fc5ac1e0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 -- 172.21.15.46:0/2854114705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ac143b30 con 0x7fc594003ef0 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5ac144bc0 0x7fc5ac1e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5aa59c640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5ac1478b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000f460 tx=0x7fc5a000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc59affd640 1 -- 172.21.15.46:0/2854114705 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a000e040 con 0x7fc594003ef0 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc59affd640 1 -- 172.21.15.46:0/2854114705 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a000f040 con 0x7fc594003ef0 2023-12-16T21:43:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ac143e10 con 0x7fc594003ef0 2023-12-16T21:43:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc59affd640 1 -- 172.21.15.46:0/2854114705 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0015a80 con 0x7fc594003ef0 2023-12-16T21:43:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.779+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ac143750 con 0x7fc594003ef0 2023-12-16T21:43:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.783+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ac0679e0 con 0x7fc594003ef0 2023-12-16T21:43:36.787 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.783+0000 7fc59affd640 1 -- 172.21.15.46:0/2854114705 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fc5a0015be0 con 0x7fc594003ef0 2023-12-16T21:43:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.783+0000 7fc59affd640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc58c066470 0x7fc58c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.783+0000 7fc59affd640 1 -- 172.21.15.46:0/2854114705 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5a0084eb0 con 0x7fc594003ef0 2023-12-16T21:43:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.783+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc58c066470 0x7fc58c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.783+0000 7fc5a9d9b640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc58c066470 0x7fc58c068930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0ab000 tx=0x7fc59c0a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:36.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.787+0000 7fc59affd640 1 -- 172.21.15.46:0/2854114705 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fc5a00523d0 con 0x7fc594003ef0 2023-12-16T21:43:36.947 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.943+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5ac0688f0 con 0x7fc594003ef0 2023-12-16T21:43:36.948 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.943+0000 7fc59affd640 1 -- 172.21.15.46:0/2854114705 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fc5a00559f0 con 0x7fc594003ef0 2023-12-16T21:43:36.948 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:36.949 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:21.267009+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:43:17.887506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-16T21:43:18.612695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:36.950 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc58c066470 msgr2=0x7fc58c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc58c066470 0x7fc58c068930 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0ab000 tx=0x7fc59c0a7070 comp rx=0 tx=0).stop 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 msgr2=0x7fc5ac1478b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5ac1478b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000f460 tx=0x7fc5a000f920 comp rx=0 tx=0).stop 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 shutdown_connections 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc58c066470 0x7fc58c068930 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc5ac1e1340 0x7fc5ac136ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc5ac144bc0 0x7fc5ac1e0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 --2- 172.21.15.46:0/2854114705 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc594003ef0 0x7fc5ac1478b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 >> 172.21.15.46:0/2854114705 conn(0x7fc5ac0b48e0 msgr2=0x7fc5ac13c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 shutdown_connections 2023-12-16T21:43:36.951 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:36.947+0000 7fc5b0c73640 1 -- 172.21.15.46:0/2854114705 wait complete. 2023-12-16T21:43:36.968 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-16T21:43:36.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:37.128 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.123+0000 7f9900790640 1 Processor -- start 2023-12-16T21:43:37.129 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.123+0000 7f9900790640 1 -- start start 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f9900790640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98f80b3fd0 0x7f98f813e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f9900790640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f98f80b0340 0x7f98f813e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f9900790640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f80af2e0 con 0x7f98f806c920 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f9900790640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f80b1f50 con 0x7f98f80b0340 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f9900790640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f813eda0 con 0x7f98f80b3fd0 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fe52c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98f806c920 0x7f98f80afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56128/0 (socket says 172.21.15.46:56128) 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fe52c640 1 -- 172.21.15.46:0/2592633357 learned_addr learned my addr 172.21.15.46:0/2592633357 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fed2d640 1 --2- 172.21.15.46:0/2592633357 >> v2:172.21.15.46:3300/0 conn(0x7f98f80b0340 0x7f98f813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fd52a640 1 -- 172.21.15.46:0/2592633357 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1667388197 0 0) 0x7f98f80af2e0 con 0x7f98f806c920 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fdd2b640 1 --2- 172.21.15.46:0/2592633357 >> v2:172.21.15.161:3300/0 conn(0x7f98f80b3fd0 0x7f98f813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.130 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fd52a640 1 -- 172.21.15.46:0/2592633357 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98e4003530 con 0x7f98f806c920 2023-12-16T21:43:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fed2d640 1 -- 172.21.15.46:0/2592633357 >> v2:172.21.15.161:3300/0 conn(0x7f98f80b3fd0 msgr2=0x7f98f813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fed2d640 1 --2- 172.21.15.46:0/2592633357 >> v2:172.21.15.161:3300/0 conn(0x7f98f80b3fd0 0x7f98f813e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fed2d640 1 -- 172.21.15.46:0/2592633357 >> v1:172.21.15.46:6789/0 conn(0x7f98f806c920 legacy=0x7f98f80afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.131 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fed2d640 1 -- 172.21.15.46:0/2592633357 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f806cd00 con 0x7f98f80b0340 2023-12-16T21:43:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fed2d640 1 --2- 172.21.15.46:0/2592633357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f80b0340 0x7f98f813e860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b15f0 tx=0x7f98f001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb177c31b960819 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fd52a640 1 -- 172.21.15.46:0/2592633357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00a5850 con 0x7f98f80b0340 2023-12-16T21:43:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fd52a640 1 -- 172.21.15.46:0/2592633357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f00a59b0 con 0x7f98f80b0340 2023-12-16T21:43:37.132 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.127+0000 7f98fd52a640 1 -- 172.21.15.46:0/2592633357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00a5d40 con 0x7f98f80b0340 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- 172.21.15.46:0/2592633357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f80b0340 msgr2=0x7f98f813e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 --2- 172.21.15.46:0/2592633357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f80b0340 0x7f98f813e860 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f98f80b15f0 tx=0x7f98f001c880 comp rx=0 tx=0).stop 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- 172.21.15.46:0/2592633357 shutdown_connections 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 --2- 172.21.15.46:0/2592633357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f80b0340 0x7f98f813e860 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 --2- 172.21.15.46:0/2592633357 >> v2:172.21.15.161:3300/0 conn(0x7f98f80b3fd0 0x7f98f813e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- 172.21.15.46:0/2592633357 >> 172.21.15.46:0/2592633357 conn(0x7f98f80b48e0 msgr2=0x7f98f80b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- 172.21.15.46:0/2592633357 shutdown_connections 2023-12-16T21:43:37.133 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- 172.21.15.46:0/2592633357 wait complete. 2023-12-16T21:43:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 Processor -- start 2023-12-16T21:43:37.134 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- start start 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f806c920 0x7f98f8145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98f8142480 0x7f98f813f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f8130560 0x7f98f81309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8142150 con 0x7f98f8130560 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f81428e0 con 0x7f98f806c920 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f8141920 con 0x7f98f8142480 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f8130560 0x7f98f81309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.135 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f8130560 0x7f98f81309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50372/0 (socket says 172.21.15.46:50372) 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 -- 172.21.15.46:0/3303613603 learned_addr learned my addr 172.21.15.46:0/3303613603 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fe52c640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f806c920 0x7f98f8145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fdd2b640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98f8142480 0x7f98f813f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 -- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98f8142480 msgr2=0x7f98f813f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98f8142480 0x7f98f813f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 -- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f806c920 msgr2=0x7f98f8145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f806c920 0x7f98f8145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 -- 172.21.15.46:0/3303613603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f0009a70 con 0x7f98f8130560 2023-12-16T21:43:37.136 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98fed2d640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f8130560 0x7f98f81309e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f98f00b0040 tx=0x7f98f0019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98eeffd640 1 -- 172.21.15.46:0/3303613603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00ad640 con 0x7f98f8130560 2023-12-16T21:43:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98eeffd640 1 -- 172.21.15.46:0/3303613603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f00ad7a0 con 0x7f98f8130560 2023-12-16T21:43:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f8140ed0 con 0x7f98f8130560 2023-12-16T21:43:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f98eeffd640 1 -- 172.21.15.46:0/3303613603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f00adb50 con 0x7f98f8130560 2023-12-16T21:43:37.137 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.131+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f8141140 con 0x7f98f8130560 2023-12-16T21:43:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.135+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f80679e0 con 0x7f98f8130560 2023-12-16T21:43:37.138 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.135+0000 7f98eeffd640 1 -- 172.21.15.46:0/3303613603 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f98f00b4050 con 0x7f98f8130560 2023-12-16T21:43:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.135+0000 7f98eeffd640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f98dc066470 0x7f98dc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.140 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.135+0000 7f98eeffd640 1 -- 172.21.15.46:0/3303613603 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98f011ec30 con 0x7f98f8130560 2023-12-16T21:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.135+0000 7f98fe52c640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f98dc066470 0x7f98dc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.135+0000 7f98fe52c640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f98dc066470 0x7f98dc068930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f98f40011f0 tx=0x7f98f4007d40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.141 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.139+0000 7f98eeffd640 1 -- 172.21.15.46:0/3303613603 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f98f00ec170 con 0x7f98f8130560 2023-12-16T21:43:37.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.323+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98f80688f0 con 0x7f98f8130560 2023-12-16T21:43:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f98eeffd640 1 -- 172.21.15.46:0/3303613603 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3359 (secure 0 0 0) 0x7f98f00b2d30 con 0x7f98f8130560 2023-12-16T21:43:37.329 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:37.329 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3332084711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3332084711},{"type":"v1","addr":"172.21.15.161:6835","nonce":3332084711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:43:19.618214+0000","modified":"2023-12-16T21:43:21.693210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/3724256427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3724256427},{"type":"v1","addr":"172.21.15.46:6833","nonce":3724256427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-16T21:43:37.329 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f98dc066470 msgr2=0x7f98dc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f98dc066470 0x7f98dc068930 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f98f40011f0 tx=0x7f98f4007d40 comp rx=0 tx=0).stop 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f8130560 msgr2=0x7f98f81309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f8130560 0x7f98f81309e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f98f00b0040 tx=0x7f98f0019620 comp rx=0 tx=0).stop 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 shutdown_connections 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f98dc066470 0x7f98dc068930 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f98f8130560 0x7f98f81309e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98f8142480 0x7f98f813f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 --2- 172.21.15.46:0/3303613603 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98f806c920 0x7f98f8145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 >> 172.21.15.46:0/3303613603 conn(0x7f98f80b48e0 msgr2=0x7f98f8133e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 shutdown_connections 2023-12-16T21:43:37.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.327+0000 7f9900790640 1 -- 172.21.15.46:0/3303613603 wait complete. 2023-12-16T21:43:37.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff486e08640 1 Processor -- start 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff486e08640 1 -- start start 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff486e08640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff48006c920 0x7ff4800afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff486e08640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff48009dd50 0x7ff48013ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff486e08640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4800af2e0 con 0x7ff4800b3fd0 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff486e08640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4800b1f50 con 0x7ff48006c920 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff486e08640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48006cd00 con 0x7ff48009dd50 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff484ba4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff48006c920 0x7ff4800afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff484ba4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7ff48006c920 0x7ff4800afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50386/0 (socket says 172.21.15.46:50386) 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.451+0000 7ff484ba4640 1 -- 172.21.15.46:0/2337375433 learned_addr learned my addr 172.21.15.46:0/2337375433 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff4853a5640 1 --2- 172.21.15.46:0/2337375433 >> v2:172.21.15.161:3300/0 conn(0x7ff48009dd50 0x7ff48013ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.458 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff484ba4640 1 -- 172.21.15.46:0/2337375433 >> v2:172.21.15.161:3300/0 conn(0x7ff48009dd50 msgr2=0x7ff48013ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:37.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff484ba4640 1 --2- 172.21.15.46:0/2337375433 >> v2:172.21.15.161:3300/0 conn(0x7ff48009dd50 0x7ff48013ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff484ba4640 1 -- 172.21.15.46:0/2337375433 >> v1:172.21.15.46:6789/0 conn(0x7ff4800b3fd0 legacy=0x7ff48013e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff47f7fe640 1 -- 172.21.15.46:0/2337375433 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103182694 0 0) 0x7ff4800af2e0 con 0x7ff4800b3fd0 2023-12-16T21:43:37.459 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff484ba4640 1 -- 172.21.15.46:0/2337375433 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4800af530 con 0x7ff48006c920 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff484ba4640 1 --2- 172.21.15.46:0/2337375433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48006c920 0x7ff4800afc30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4800b0f90 tx=0x7ff47801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa57b2459d95d482 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff47f7fe640 1 -- 172.21.15.46:0/2337375433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780ab600 con 0x7ff48006c920 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff47f7fe640 1 -- 172.21.15.46:0/2337375433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4780a7050 con 0x7ff48006c920 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff47f7fe640 1 -- 172.21.15.46:0/2337375433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780ab980 con 0x7ff48006c920 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 -- 172.21.15.46:0/2337375433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48006c920 msgr2=0x7ff4800afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 --2- 172.21.15.46:0/2337375433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48006c920 0x7ff4800afc30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7ff4800b0f90 tx=0x7ff47801c470 comp rx=0 tx=0).stop 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 -- 172.21.15.46:0/2337375433 shutdown_connections 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 --2- 172.21.15.46:0/2337375433 >> v2:172.21.15.161:3300/0 conn(0x7ff48009dd50 0x7ff48013ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 --2- 172.21.15.46:0/2337375433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff48006c920 0x7ff4800afc30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 -- 172.21.15.46:0/2337375433 >> 172.21.15.46:0/2337375433 conn(0x7ff4800b48e0 msgr2=0x7ff4800b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 -- 172.21.15.46:0/2337375433 shutdown_connections 2023-12-16T21:43:37.460 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 -- 172.21.15.46:0/2337375433 wait complete. 2023-12-16T21:43:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 Processor -- start 2023-12-16T21:43:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.455+0000 7ff486e08640 1 -- start start 2023-12-16T21:43:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff48006c920 0x7ff480144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.462 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48009dd50 0x7ff4801439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff480143f30 0x7ff4801e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4800719f0 con 0x7ff480143f30 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48006f5b0 con 0x7ff48009dd50 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4801443c0 con 0x7ff48006c920 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff480143f30 0x7ff4801e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff480143f30 0x7ff4801e20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50394/0 (socket says 172.21.15.46:50394) 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 -- 172.21.15.46:0/1880990234 learned_addr learned my addr 172.21.15.46:0/1880990234 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff47ffff640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48009dd50 0x7ff4801439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff484ba4640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff48006c920 0x7ff480144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 -- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff48006c920 msgr2=0x7ff480144eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff48006c920 0x7ff480144eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 -- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48009dd50 msgr2=0x7ff4801439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48009dd50 0x7ff4801439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.463 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 -- 172.21.15.46:0/1880990234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780094d0 con 0x7ff480143f30 2023-12-16T21:43:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff4853a5640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff480143f30 0x7ff4801e20b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff470007e60 tx=0x7ff47000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff47d7fa640 1 -- 172.21.15.46:0/1880990234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470014420 con 0x7ff480143f30 2023-12-16T21:43:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff47d7fa640 1 -- 172.21.15.46:0/1880990234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff470014580 con 0x7ff480143f30 2023-12-16T21:43:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff47d7fa640 1 -- 172.21.15.46:0/1880990234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff470014930 con 0x7ff480143f30 2023-12-16T21:43:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48006fa90 con 0x7ff480143f30 2023-12-16T21:43:37.464 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff480139220 con 0x7ff480143f30 2023-12-16T21:43:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4800679e0 con 0x7ff480143f30 2023-12-16T21:43:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff47d7fa640 1 -- 172.21.15.46:0/1880990234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7ff470014a90 con 0x7ff480143f30 2023-12-16T21:43:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff47d7fa640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff460066470 0x7ff460068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff47d7fa640 1 -- 172.21.15.46:0/1880990234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff470084820 con 0x7ff480143f30 2023-12-16T21:43:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.459+0000 7ff484ba4640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff460066470 0x7ff460068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.465 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.463+0000 7ff484ba4640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff460066470 0x7ff460068930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff4780ad000 tx=0x7ff478008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.468 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.463+0000 7ff47d7fa640 1 -- 172.21.15.46:0/1880990234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7ff470051c90 con 0x7ff480143f30 2023-12-16T21:43:37.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.651+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff480138da0 con 0x7ff480143f30 2023-12-16T21:43:37.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.651+0000 7ff47d7fa640 1 -- 172.21.15.46:0/1880990234 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3359 (secure 0 0 0) 0x7ff480138da0 con 0x7ff480143f30 2023-12-16T21:43:37.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:37.656 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3332084711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3332084711},{"type":"v1","addr":"172.21.15.161:6835","nonce":3332084711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-16T21:43:19.618214+0000","modified":"2023-12-16T21:43:21.693210+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.46:6833/3724256427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":3724256427},{"type":"v1","addr":"172.21.15.46:6833","nonce":3724256427}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-16T21:43:37.656 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2023-12-16T21:43:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff460066470 msgr2=0x7ff460068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff460066470 0x7ff460068930 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff4780ad000 tx=0x7ff478008040 comp rx=0 tx=0).stop 2023-12-16T21:43:37.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff480143f30 msgr2=0x7ff4801e20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff480143f30 0x7ff4801e20b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7ff470007e60 tx=0x7ff47000e4b0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 shutdown_connections 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff460066470 0x7ff460068930 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff480143f30 0x7ff4801e20b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff48009dd50 0x7ff4801439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 --2- 172.21.15.46:0/1880990234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff48006c920 0x7ff480144eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 >> 172.21.15.46:0/1880990234 conn(0x7ff4800b48e0 msgr2=0x7ff48013d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 shutdown_connections 2023-12-16T21:43:37.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.655+0000 7ff486e08640 1 -- 172.21.15.46:0/1880990234 wait complete. 2023-12-16T21:43:37.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.779+0000 7fdf1d6f6640 1 Processor -- start 2023-12-16T21:43:37.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.779+0000 7fdf1d6f6640 1 -- start start 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf1d6f6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf1806c920 0x7fdf180afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf1d6f6640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdf1809dd50 0x7fdf1813ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf1d6f6640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf180af2e0 con 0x7fdf180b3fd0 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf1d6f6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf180b1f50 con 0x7fdf1806c920 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf1d6f6640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1806cd00 con 0x7fdf1809dd50 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf1806c920 0x7fdf180afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdf1806c920 0x7fdf180afc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50396/0 (socket says 172.21.15.46:50396) 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 -- 172.21.15.46:0/2006762741 learned_addr learned my addr 172.21.15.46:0/2006762741 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 -- 172.21.15.46:0/2006762741 >> v2:172.21.15.161:3300/0 conn(0x7fdf1809dd50 msgr2=0x7fdf1813ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 --2- 172.21.15.46:0/2006762741 >> v2:172.21.15.161:3300/0 conn(0x7fdf1809dd50 0x7fdf1813ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 -- 172.21.15.46:0/2006762741 >> v1:172.21.15.161:6789/0 conn(0x7fdf180b3fd0 legacy=0x7fdf1813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:37.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 -- 172.21.15.46:0/2006762741 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180af530 con 0x7fdf1806c920 2023-12-16T21:43:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf16ffd640 1 --2- 172.21.15.46:0/2006762741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1806c920 0x7fdf180afc30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf180b0f90 tx=0x7fdf0c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec000fa8403dc79 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf15ffb640 1 -- 172.21.15.46:0/2006762741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0ab600 con 0x7fdf1806c920 2023-12-16T21:43:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf15ffb640 1 -- 172.21.15.46:0/2006762741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c0a7050 con 0x7fdf1806c920 2023-12-16T21:43:37.788 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.783+0000 7fdf15ffb640 1 -- 172.21.15.46:0/2006762741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0ab980 con 0x7fdf1806c920 2023-12-16T21:43:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/2006762741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1806c920 msgr2=0x7fdf180afc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/2006762741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1806c920 0x7fdf180afc30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf180b0f90 tx=0x7fdf0c01c430 comp rx=0 tx=0).stop 2023-12-16T21:43:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/2006762741 shutdown_connections 2023-12-16T21:43:37.789 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/2006762741 >> v2:172.21.15.161:3300/0 conn(0x7fdf1809dd50 0x7fdf1813ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/2006762741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf1806c920 0x7fdf180afc30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/2006762741 >> 172.21.15.46:0/2006762741 conn(0x7fdf180b48e0 msgr2=0x7fdf180b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/2006762741 shutdown_connections 2023-12-16T21:43:37.790 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/2006762741 wait complete. 2023-12-16T21:43:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 Processor -- start 2023-12-16T21:43:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- start start 2023-12-16T21:43:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf1806c920 0x7fdf18070f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1809dd50 0x7fdf18099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf181e3060 0x7fdf1813bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf181e2bd0 con 0x7fdf181e3060 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18071540 con 0x7fdf1809dd50 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf181e2860 con 0x7fdf1806c920 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf181e3060 0x7fdf1813bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf181e3060 0x7fdf1813bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50404/0 (socket says 172.21.15.46:50404) 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 -- 172.21.15.46:0/366434742 learned_addr learned my addr 172.21.15.46:0/366434742 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf167fc640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1809dd50 0x7fdf18099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf16ffd640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf1806c920 0x7fdf18070f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 -- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf1806c920 msgr2=0x7fdf18070f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf1806c920 0x7fdf18070f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 -- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1809dd50 msgr2=0x7fdf18099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1809dd50 0x7fdf18099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 -- 172.21.15.46:0/366434742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0094d0 con 0x7fdf181e3060 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf177fe640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf181e3060 0x7fdf1813bb80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b9e0 tx=0x7fdf1000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdeff7fe640 1 -- 172.21.15.46:0/366434742 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1000cb20 con 0x7fdf181e3060 2023-12-16T21:43:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdeff7fe640 1 -- 172.21.15.46:0/366434742 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1000cc80 con 0x7fdf181e3060 2023-12-16T21:43:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1813fe80 con 0x7fdf181e3060 2023-12-16T21:43:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdeff7fe640 1 -- 172.21.15.46:0/366434742 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf10014680 con 0x7fdf181e3060 2023-12-16T21:43:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf18099fb0 con 0x7fdf181e3060 2023-12-16T21:43:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.787+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1809b370 con 0x7fdf181e3060 2023-12-16T21:43:37.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.791+0000 7fdeff7fe640 1 -- 172.21.15.46:0/366434742 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fdf1001d3b0 con 0x7fdf181e3060 2023-12-16T21:43:37.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.791+0000 7fdeff7fe640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf040664a0 0x7fdf04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:37.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.791+0000 7fdeff7fe640 1 -- 172.21.15.46:0/366434742 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf10083d00 con 0x7fdf181e3060 2023-12-16T21:43:37.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.791+0000 7fdf16ffd640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf040664a0 0x7fdf04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:37.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.791+0000 7fdf16ffd640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf040664a0 0x7fdf04068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0ad000 tx=0x7fdf0c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:37.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.795+0000 7fdeff7fe640 1 -- 172.21.15.46:0/366434742 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fdf10051210 con 0x7fdf181e3060 2023-12-16T21:43:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.943+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf180679e0 con 0x7fdf181e3060 2023-12-16T21:43:37.949 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdeff7fe640 1 -- 172.21.15.46:0/366434742 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fdf10019090 con 0x7fdf181e3060 2023-12-16T21:43:37.949 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:37.950 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":30,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:21.267009+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-16T21:43:17.887506+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-16T21:43:18.612695+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf040664a0 msgr2=0x7fdf04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf040664a0 0x7fdf04068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0ad000 tx=0x7fdf0c0a6000 comp rx=0 tx=0).stop 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf181e3060 msgr2=0x7fdf1813bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf181e3060 0x7fdf1813bb80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf1000b9e0 tx=0x7fdf1000bea0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 shutdown_connections 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdf040664a0 0x7fdf04068960 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf181e3060 0x7fdf1813bb80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdf1809dd50 0x7fdf18099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 --2- 172.21.15.46:0/366434742 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdf1806c920 0x7fdf18070f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 >> 172.21.15.46:0/366434742 conn(0x7fdf180b48e0 msgr2=0x7fdf18144590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 shutdown_connections 2023-12-16T21:43:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:37.947+0000 7fdf1d6f6640 1 -- 172.21.15.46:0/366434742 wait complete. 2023-12-16T21:43:37.967 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-16T21:43:38.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.075+0000 7ff559441640 1 Processor -- start 2023-12-16T21:43:38.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.075+0000 7ff559441640 1 -- start start 2023-12-16T21:43:38.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff559441640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7ff5540b7b60 0x7ff554147b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff559441640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540ba8d0 con 0x7ff55413ea10 2023-12-16T21:43:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff559441640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540b7ff0 con 0x7ff5540ba4f0 2023-12-16T21:43:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff559441640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540b1500 con 0x7ff5540b7b60 2023-12-16T21:43:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5537fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7ff55413ea10 0x7ff55414edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56144/0 (socket says 172.21.15.46:56144) 2023-12-16T21:43:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5537fe640 1 -- 172.21.15.46:0/489487309 learned_addr learned my addr 172.21.15.46:0/489487309 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff54bfff640 1 --2- 172.21.15.46:0/489487309 >> v2:172.21.15.161:3300/0 conn(0x7ff5540b7b60 0x7ff554147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811999145 0 0) 0x7ff5540ba8d0 con 0x7ff55413ea10 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff540003530 con 0x7ff55413ea10 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1059219966 0 0) 0x7ff540003530 con 0x7ff55413ea10 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 >> v2:172.21.15.161:3300/0 conn(0x7ff5540b7b60 msgr2=0x7ff554147b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 --2- 172.21.15.46:0/489487309 >> v2:172.21.15.161:3300/0 conn(0x7ff5540b7b60 0x7ff554147b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 >> v1:172.21.15.161:6789/0 conn(0x7ff5540ba4f0 legacy=0x7ff55413e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540b82d0 con 0x7ff55413ea10 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7ff54c001a00 con 0x7ff55413ea10 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 >> v1:172.21.15.46:6789/0 conn(0x7ff55413ea10 legacy=0x7ff55414edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff540003980 0x7ff540003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5400047d0 0x7ff540004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff540005190 0x7ff54000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54000c000 con 0x7ff5400047d0 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54000c170 con 0x7ff540005190 2023-12-16T21:43:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54000c330 con 0x7ff540003980 2023-12-16T21:43:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff54bfff640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5400047d0 0x7ff540004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff54bfff640 1 -- 172.21.15.46:0/489487309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff540003980 msgr2=0x7ff540003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff54bfff640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff540003980 0x7ff540003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff54bfff640 1 -- 172.21.15.46:0/489487309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff540005190 msgr2=0x7ff54000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff54bfff640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff540005190 0x7ff54000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff54bfff640 1 -- 172.21.15.46:0/489487309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540003530 con 0x7ff5400047d0 2023-12-16T21:43:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.079+0000 7ff5537fe640 1 -- 172.21.15.46:0/489487309 reap_dead start 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff54bfff640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5400047d0 0x7ff540004c50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009ae0 tx=0x7ff53c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e98242a127e4ff3 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0a5640 con 0x7ff5400047d0 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c0a57a0 con 0x7ff5400047d0 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5527fc640 1 -- 172.21.15.46:0/489487309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0a5b50 con 0x7ff5400047d0 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- 172.21.15.46:0/489487309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5400047d0 msgr2=0x7ff540004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5400047d0 0x7ff540004c50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009ae0 tx=0x7ff53c01ca50 comp rx=0 tx=0).stop 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- 172.21.15.46:0/489487309 shutdown_connections 2023-12-16T21:43:38.088 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 --2- 172.21.15.46:0/489487309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5400047d0 0x7ff540004c50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- 172.21.15.46:0/489487309 >> 172.21.15.46:0/489487309 conn(0x7ff55406a980 msgr2=0x7ff55406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- 172.21.15.46:0/489487309 shutdown_connections 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- 172.21.15.46:0/489487309 wait complete. 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 Processor -- start 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- start start 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5400047d0 0x7ff554146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55406ba70 0x7ff55406bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5541ded80 0x7ff5541df200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55406c430 con 0x7ff5541ded80 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55413bd50 con 0x7ff55406ba70 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55413c340 con 0x7ff5400047d0 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5541ded80 0x7ff5541df200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.089 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5541ded80 0x7ff5541df200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50418/0 (socket says 172.21.15.46:50418) 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 -- 172.21.15.46:0/57717679 learned_addr learned my addr 172.21.15.46:0/57717679 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff552ffd640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5400047d0 0x7ff554146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff54bfff640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55406ba70 0x7ff55406bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 -- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5400047d0 msgr2=0x7ff554146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5400047d0 0x7ff554146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 -- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55406ba70 msgr2=0x7ff55406bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55406ba70 0x7ff55406bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 -- 172.21.15.46:0/57717679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55413c620 con 0x7ff5541ded80 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff54bfff640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55406ba70 0x7ff55406bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff5537fe640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5541ded80 0x7ff5541df200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff54c004c00 tx=0x7ff54c00fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff54b7fe640 1 -- 172.21.15.46:0/57717679 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00da20 con 0x7ff5541ded80 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff54b7fe640 1 -- 172.21.15.46:0/57717679 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c00dba0 con 0x7ff5541ded80 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55413d670 con 0x7ff5541ded80 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff54b7fe640 1 -- 172.21.15.46:0/57717679 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00feb0 con 0x7ff5541ded80 2023-12-16T21:43:38.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.083+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55413b8a0 con 0x7ff5541ded80 2023-12-16T21:43:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.087+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55413c1a0 con 0x7ff5541ded80 2023-12-16T21:43:38.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.087+0000 7ff54b7fe640 1 -- 172.21.15.46:0/57717679 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7ff54c00a0d0 con 0x7ff5541ded80 2023-12-16T21:43:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.087+0000 7ff54b7fe640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff5340664d0 0x7ff534068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.087+0000 7ff54b7fe640 1 -- 172.21.15.46:0/57717679 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff54c086580 con 0x7ff5541ded80 2023-12-16T21:43:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.087+0000 7ff552ffd640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff5340664d0 0x7ff534068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.087+0000 7ff552ffd640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff5340664d0 0x7ff534068990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff544001660 tx=0x7ff544009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:38.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.091+0000 7ff54b7fe640 1 -- 172.21.15.46:0/57717679 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7ff55413c1a0 con 0x7ff5541ded80 2023-12-16T21:43:38.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.279+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff55413c1a0 con 0x7ff5541ded80 2023-12-16T21:43:38.282 INFO:tasks.ceph.mon.a.smithi046.stderr:2023-12-16T21:43:38.279+0000 7f80e84dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-16T21:43:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.283+0000 7ff54b7fe640 1 -- 172.21.15.46:0/57717679 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff55413c1a0 con 0x7ff5541ded80 2023-12-16T21:43:38.285 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-16T21:43:38.289 INFO:tasks.ceph.mds.a.smithi046.stderr: -2520> 2023-12-16T21:43:24.035+0000 7fd37d85a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-16T21:43:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff5340664d0 msgr2=0x7ff534068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff5340664d0 0x7ff534068990 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff544001660 tx=0x7ff544009040 comp rx=0 tx=0).stop 2023-12-16T21:43:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5541ded80 msgr2=0x7ff5541df200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.290 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5541ded80 0x7ff5541df200 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff54c004c00 tx=0x7ff54c00fc60 comp rx=0 tx=0).stop 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 shutdown_connections 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7ff5340664d0 0x7ff534068990 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff5541ded80 0x7ff5541df200 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7ff55406ba70 0x7ff55406bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 --2- 172.21.15.46:0/57717679 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7ff5400047d0 0x7ff554146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 >> 172.21.15.46:0/57717679 conn(0x7ff55406a980 msgr2=0x7ff5540b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 shutdown_connections 2023-12-16T21:43:38.291 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.287+0000 7ff559441640 1 -- 172.21.15.46:0/57717679 wait complete. 2023-12-16T21:43:38.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-16T21:43:38.320 INFO:tasks.ceph.mds.a.smithi046.stdout:starting mds.a at 2023-12-16T21:43:38.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fc4dc640 1 Processor -- start 2023-12-16T21:43:38.415 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fc4dc640 1 -- start start 2023-12-16T21:43:38.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fc4dc640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 0x7f42f414fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fc4dc640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f40b1730 con 0x7f42f40b1eb0 2023-12-16T21:43:38.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fc4dc640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f406c7f0 con 0x7f42f40af300 2023-12-16T21:43:38.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fc4dc640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f406cd70 con 0x7f42f41451c0 2023-12-16T21:43:38.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fa278640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f42f40b1eb0 0x7f42f40aff30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:56164/0 (socket says 172.21.15.46:56164) 2023-12-16T21:43:38.416 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42fa278640 1 -- 172.21.15.46:0/898797622 learned_addr learned my addr 172.21.15.46:0/898797622 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:38.418 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.411+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257472889 0 0) 0x7f42f40b1730 con 0x7f42f40b1eb0 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 0x7f42f414fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42e0003530 con 0x7f42f40b1eb0 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118942703 0 0) 0x7f42e0003530 con 0x7f42f40b1eb0 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 msgr2=0x7f42f414fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 0x7f42f414fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 >> v1:172.21.15.161:6789/0 conn(0x7f42f40af300 legacy=0x7f42f4144ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40b2290 con 0x7f42f40b1eb0 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f42f0001680 con 0x7f42f40b1eb0 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 >> v1:172.21.15.46:6789/0 conn(0x7f42f40b1eb0 legacy=0x7f42f40aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42e00039b0 0x7f42e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 0x7f42e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f406cd70 con 0x7f42e0004d10 2023-12-16T21:43:38.419 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e000bb50 con 0x7f42e00039b0 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e000bd10 con 0x7f42f41451c0 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42fa278640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42e00039b0 0x7f42e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9a77640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 0x7f42e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 -- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 msgr2=0x7f42e00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f41451c0 0x7f42e00047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 -- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42e00039b0 msgr2=0x7f42e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42e00039b0 0x7f42e0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 -- 172.21.15.46:0/898797622 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e0003530 con 0x7f42e0004d10 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42e0005190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009f90 tx=0x7f42ec01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14528a561cdd77d server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0a5aa0 con 0x7f42e0004d10 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ec00f040 con 0x7f42e0004d10 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42f9276640 1 -- 172.21.15.46:0/898797622 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ec0a5e00 con 0x7f42e0004d10 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42fc4dc640 1 -- 172.21.15.46:0/898797622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 msgr2=0x7f42e0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42fc4dc640 1 --2- 172.21.15.46:0/898797622 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42e0005190 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f42ec009f90 tx=0x7f42ec01cc20 comp rx=0 tx=0).stop 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42faa79640 1 -- 172.21.15.46:0/898797622 reap_dead start 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42fc4dc640 1 -- 172.21.15.46:0/898797622 shutdown_connections 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42fc4dc640 1 -- 172.21.15.46:0/898797622 >> 172.21.15.46:0/898797622 conn(0x7f42f40b4900 msgr2=0x7f42f40b4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42fc4dc640 1 -- 172.21.15.46:0/898797622 shutdown_connections 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.415+0000 7f42fc4dc640 1 -- 172.21.15.46:0/898797622 wait complete. 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 Processor -- start 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 -- start start 2023-12-16T21:43:38.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42f4138a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f40ada10 0x7f42f406e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f406ecc0 0x7f42f4132e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f40aebc0 con 0x7f42e0004d10 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f406fd90 con 0x7f42f40ada10 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f41e2040 con 0x7f42f406ecc0 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42f4138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42f4138a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50470/0 (socket says 172.21.15.46:50470) 2023-12-16T21:43:38.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 -- 172.21.15.46:0/436769109 learned_addr learned my addr 172.21.15.46:0/436769109 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 -- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f406ecc0 msgr2=0x7f42f4132e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f406ecc0 0x7f42f4132e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 -- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f40ada10 msgr2=0x7f42f406e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f40ada10 0x7f42f406e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 -- 172.21.15.46:0/436769109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4070000 con 0x7f42e0004d10 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fa278640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42f4138a40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f42f0007fd0 tx=0x7f42f000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42eaffd640 1 -- 172.21.15.46:0/436769109 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00072d0 con 0x7f42e0004d10 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42eaffd640 1 -- 172.21.15.46:0/436769109 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0007430 con 0x7f42e0004d10 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f40acf50 con 0x7f42e0004d10 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42eaffd640 1 -- 172.21.15.46:0/436769109 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00047f0 con 0x7f42e0004d10 2023-12-16T21:43:38.425 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f41333a0 con 0x7f42e0004d10 2023-12-16T21:43:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f40ad190 con 0x7f42e0004d10 2023-12-16T21:43:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42eaffd640 1 -- 172.21.15.46:0/436769109 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f42f0004950 con 0x7f42e0004d10 2023-12-16T21:43:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42eaffd640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f42d8066470 0x7f42d8068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42eaffd640 1 -- 172.21.15.46:0/436769109 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f42f00839d0 con 0x7f42e0004d10 2023-12-16T21:43:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.419+0000 7f42f9a77640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f42d8066470 0x7f42d8068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:38.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.423+0000 7f42f9a77640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f42d8066470 0x7f42d8068930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f42f40b0aa0 tx=0x7f42e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:38.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.423+0000 7f42eaffd640 1 -- 172.21.15.46:0/436769109 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f42f40ad190 con 0x7f42e0004d10 2023-12-16T21:43:38.617 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:38.615+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f42f40ad190 con 0x7f42e0004d10 2023-12-16T21:43:39.297 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42eaffd640 1 -- 172.21.15.46:0/436769109 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f42f40ad190 con 0x7f42e0004d10 2023-12-16T21:43:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f42d8066470 msgr2=0x7f42d8068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f42d8066470 0x7f42d8068930 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f42f40b0aa0 tx=0x7f42e4009040 comp rx=0 tx=0).stop 2023-12-16T21:43:39.301 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 msgr2=0x7f42f4138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42f4138a40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f42f0007fd0 tx=0x7f42f000db40 comp rx=0 tx=0).stop 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 shutdown_connections 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f42d8066470 0x7f42d8068930 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f42f406ecc0 0x7f42f4132e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f42f40ada10 0x7f42f406e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 --2- 172.21.15.46:0/436769109 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f42e0004d10 0x7f42f4138a40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.295+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 >> 172.21.15.46:0/436769109 conn(0x7f42f40b4900 msgr2=0x7f42f41423e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.299+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 shutdown_connections 2023-12-16T21:43:39.302 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.299+0000 7f42fc4dc640 1 -- 172.21.15.46:0/436769109 wait complete. 2023-12-16T21:43:39.324 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-16T21:43:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a3f2b640 1 Processor -- start 2023-12-16T21:43:39.501 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a3f2b640 1 -- start start 2023-12-16T21:43:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a3f2b640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fc09c0b7190 0x7fc09c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a3f2b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc09c0b5ce0 0x7fc09c147a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a3f2b640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc09c141f60 con 0x7fc09c141400 2023-12-16T21:43:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a3f2b640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c0b6650 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.507 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a3f2b640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09c0b4940 con 0x7fc09c0b7190 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a14c6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fc09c0b5ce0 0x7fc09c147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a24c8640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fc09c141400 0x7fc09c141850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46408/0 (socket says 172.21.15.46:46408) 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a24c8640 1 -- 172.21.15.46:0/3549384758 learned_addr learned my addr 172.21.15.46:0/3549384758 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a0cc5640 1 -- 172.21.15.46:0/3549384758 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897723273 0 0) 0x7fc09c141f60 con 0x7fc09c141400 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a1cc7640 1 --2- 172.21.15.46:0/3549384758 >> v2:172.21.15.161:3300/0 conn(0x7fc09c0b7190 0x7fc09c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a0cc5640 1 -- 172.21.15.46:0/3549384758 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc084003530 con 0x7fc09c141400 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a14c6640 1 -- 172.21.15.46:0/3549384758 >> v2:172.21.15.161:3300/0 conn(0x7fc09c0b7190 msgr2=0x7fc09c0b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a14c6640 1 --2- 172.21.15.46:0/3549384758 >> v2:172.21.15.161:3300/0 conn(0x7fc09c0b7190 0x7fc09c0b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a14c6640 1 -- 172.21.15.46:0/3549384758 >> v1:172.21.15.46:6789/0 conn(0x7fc09c141400 legacy=0x7fc09c141850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a14c6640 1 -- 172.21.15.46:0/3549384758 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09c0b20f0 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.499+0000 7fc0a1cc7640 1 --2- 172.21.15.46:0/3549384758 >> v2:172.21.15.161:3300/0 conn(0x7fc09c0b7190 0x7fc09c0b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a14c6640 1 --2- 172.21.15.46:0/3549384758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c147a00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0baf80 tx=0x7fc09401c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20faf8417595aaea server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a0cc5640 1 -- 172.21.15.46:0/3549384758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940a57a0 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.508 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a0cc5640 1 -- 172.21.15.46:0/3549384758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0940a5900 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a0cc5640 1 -- 172.21.15.46:0/3549384758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0940a5c90 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3549384758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 msgr2=0x7fc09c147a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3549384758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c147a00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc09c0baf80 tx=0x7fc09401c7f0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3549384758 shutdown_connections 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3549384758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c147a00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3549384758 >> v2:172.21.15.161:3300/0 conn(0x7fc09c0b7190 0x7fc09c0b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3549384758 >> 172.21.15.46:0/3549384758 conn(0x7fc09c06a980 msgr2=0x7fc09c06ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3549384758 shutdown_connections 2023-12-16T21:43:39.509 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3549384758 wait complete. 2023-12-16T21:43:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 Processor -- start 2023-12-16T21:43:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- start start 2023-12-16T21:43:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c138830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc09c0b7190 0x7fc09c13a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09c141400 0x7fc09c1df670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c132af0 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.512 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c13b790 con 0x7fc09c141400 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.503+0000 7fc0a3f2b640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09c143fa0 con 0x7fc09c0b7190 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c138830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c138830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35452/0 (socket says 172.21.15.46:35452) 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 -- 172.21.15.46:0/3522536234 learned_addr learned my addr 172.21.15.46:0/3522536234 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a24c8640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09c141400 0x7fc09c1df670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a14c6640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc09c0b7190 0x7fc09c13a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 -- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc09c0b7190 msgr2=0x7fc09c13a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc09c0b7190 0x7fc09c13a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 -- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09c141400 msgr2=0x7fc09c1df670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09c141400 0x7fc09c1df670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 -- 172.21.15.46:0/3522536234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094009a70 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a24c8640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09c141400 0x7fc09c1df670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a14c6640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc09c0b7190 0x7fc09c13a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:39.513 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a1cc7640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c138830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c7c0 tx=0x7fc09800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0927fc640 1 -- 172.21.15.46:0/3522536234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098007a30 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0927fc640 1 -- 172.21.15.46:0/3522536234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc098007b90 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0927fc640 1 -- 172.21.15.46:0/3522536234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800ce70 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc09c139920 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc09c143450 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09c0689d0 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0927fc640 1 -- 172.21.15.46:0/3522536234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fc0980113f0 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0927fc640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc088066470 0x7fc088068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0927fc640 1 -- 172.21.15.46:0/3522536234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc0980843f0 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.507+0000 7fc0a14c6640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc088066470 0x7fc088068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:39.514 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.511+0000 7fc0a14c6640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc088066470 0x7fc088068930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0940af000 tx=0x7fc0940a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:39.516 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.511+0000 7fc0927fc640 1 -- 172.21.15.46:0/3522536234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fc098051800 con 0x7fc09c0b5ce0 2023-12-16T21:43:39.672 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:39.667+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc09c13b990 con 0x7fc09c0b5ce0 2023-12-16T21:43:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.487+0000 7fc0927fc640 1 -- 172.21.15.46:0/3522536234 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc098054e20 con 0x7fc09c0b5ce0 2023-12-16T21:43:40.492 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2023-12-16T21:43:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc088066470 msgr2=0x7fc088068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc088066470 0x7fc088068930 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc0940af000 tx=0x7fc0940a7070 comp rx=0 tx=0).stop 2023-12-16T21:43:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 msgr2=0x7fc09c138830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c138830 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc09800c7c0 tx=0x7fc09800cc80 comp rx=0 tx=0).stop 2023-12-16T21:43:40.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 shutdown_connections 2023-12-16T21:43:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fc088066470 0x7fc088068930 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fc09c141400 0x7fc09c1df670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fc09c0b7190 0x7fc09c13a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 --2- 172.21.15.46:0/3522536234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc09c0b5ce0 0x7fc09c138830 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 >> 172.21.15.46:0/3522536234 conn(0x7fc09c06a980 msgr2=0x7fc09c070930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 shutdown_connections 2023-12-16T21:43:40.496 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.491+0000 7fc0a3f2b640 1 -- 172.21.15.46:0/3522536234 wait complete. 2023-12-16T21:43:40.523 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-16T21:43:40.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.655+0000 7fe3e1b69640 1 Processor -- start 2023-12-16T21:43:40.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.655+0000 7fe3e1b69640 1 -- start start 2023-12-16T21:43:40.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3dc0736b0 0x7fe3dc09c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0aee80 0x7fe3dc0b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc09daf0 con 0x7fe3dc0b4bb0 2023-12-16T21:43:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc09bd90 con 0x7fe3dc0736b0 2023-12-16T21:43:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0ae9f0 con 0x7fe3dc0aee80 2023-12-16T21:43:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3daffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe3dc0b4bb0 0x7fe3dc0b2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46418/0 (socket says 172.21.15.46:46418) 2023-12-16T21:43:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3db7fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fe3dc0736b0 0x7fe3dc09c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:40.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3daffd640 1 -- 172.21.15.46:0/2777909811 learned_addr learned my addr 172.21.15.46:0/2777909811 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3dbfff640 1 --2- 172.21.15.46:0/2777909811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0aee80 0x7fe3dc0b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3db7fe640 1 -- 172.21.15.46:0/2777909811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0aee80 msgr2=0x7fe3dc0b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3db7fe640 1 --2- 172.21.15.46:0/2777909811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0aee80 0x7fe3dc0b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3db7fe640 1 -- 172.21.15.46:0/2777909811 >> v1:172.21.15.46:6789/0 conn(0x7fe3dc0b4bb0 legacy=0x7fe3dc0b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3da7fc640 1 -- 172.21.15.46:0/2777909811 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300555047 0 0) 0x7fe3d0001120 con 0x7fe3dc0b4bb0 2023-12-16T21:43:40.662 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3db7fe640 1 -- 172.21.15.46:0/2777909811 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc073a90 con 0x7fe3dc0736b0 2023-12-16T21:43:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3db7fe640 1 --2- 172.21.15.46:0/2777909811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0736b0 0x7fe3dc09c420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097c0 tx=0x7fe3cc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b207a0c1605c07 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3da7fc640 1 -- 172.21.15.46:0/2777909811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0ad660 con 0x7fe3dc0736b0 2023-12-16T21:43:40.663 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3da7fc640 1 -- 172.21.15.46:0/2777909811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3cc0ad7c0 con 0x7fe3dc0736b0 2023-12-16T21:43:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3da7fc640 1 -- 172.21.15.46:0/2777909811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3cc0adb50 con 0x7fe3dc0736b0 2023-12-16T21:43:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- 172.21.15.46:0/2777909811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0736b0 msgr2=0x7fe3dc09c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:40.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/2777909811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0736b0 0x7fe3dc09c420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0097c0 tx=0x7fe3cc01c3c0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- 172.21.15.46:0/2777909811 shutdown_connections 2023-12-16T21:43:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/2777909811 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0aee80 0x7fe3dc0b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/2777909811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0736b0 0x7fe3dc09c420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- 172.21.15.46:0/2777909811 >> 172.21.15.46:0/2777909811 conn(0x7fe3dc0b55e0 msgr2=0x7fe3dc099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- 172.21.15.46:0/2777909811 shutdown_connections 2023-12-16T21:43:40.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.659+0000 7fe3e1b69640 1 -- 172.21.15.46:0/2777909811 wait complete. 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 Processor -- start 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 -- start start 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0736b0 0x7fe3dc0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0aee80 0x7fe3dc142ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc0b4bb0 0x7fe3dc071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0b0550 con 0x7fe3dc0aee80 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0b0d90 con 0x7fe3dc0b4bb0 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc06f220 con 0x7fe3dc0736b0 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0aee80 0x7fe3dc142ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0aee80 0x7fe3dc142ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35476/0 (socket says 172.21.15.46:35476) 2023-12-16T21:43:40.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 -- 172.21.15.46:0/410006926 learned_addr learned my addr 172.21.15.46:0/410006926 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3dbfff640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc0b4bb0 0x7fe3dc071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3db7fe640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0736b0 0x7fe3dc0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 -- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0736b0 msgr2=0x7fe3dc0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0736b0 0x7fe3dc0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 -- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc0b4bb0 msgr2=0x7fe3dc071990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc0b4bb0 0x7fe3dc071990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 -- 172.21.15.46:0/410006926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc009420 con 0x7fe3dc0aee80 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3dbfff640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc0b4bb0 0x7fe3dc071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3db7fe640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0736b0 0x7fe3dc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3daffd640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0aee80 0x7fe3dc142ae0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c3e0 tx=0x7fe3d000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3c3fff640 1 -- 172.21.15.46:0/410006926 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d00074e0 con 0x7fe3dc0aee80 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3c3fff640 1 -- 172.21.15.46:0/410006926 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d0007660 con 0x7fe3dc0aee80 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc0b0740 con 0x7fe3dc0aee80 2023-12-16T21:43:40.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3c3fff640 1 -- 172.21.15.46:0/410006926 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0004a30 con 0x7fe3dc0aee80 2023-12-16T21:43:40.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc1444c0 con 0x7fe3dc0aee80 2023-12-16T21:43:40.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc067fa0 con 0x7fe3dc0aee80 2023-12-16T21:43:40.669 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.663+0000 7fe3c3fff640 1 -- 172.21.15.46:0/410006926 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fe3d0004b90 con 0x7fe3dc0aee80 2023-12-16T21:43:40.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.667+0000 7fe3c3fff640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3c80664a0 0x7fe3c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:40.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.667+0000 7fe3c3fff640 1 -- 172.21.15.46:0/410006926 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe3d0083880 con 0x7fe3dc0aee80 2023-12-16T21:43:40.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.667+0000 7fe3db7fe640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3c80664a0 0x7fe3c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:40.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.667+0000 7fe3db7fe640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3c80664a0 0x7fe3c8068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0af000 tx=0x7fe3cc0a7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:40.673 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.671+0000 7fe3c3fff640 1 -- 172.21.15.46:0/410006926 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fe3d0054980 con 0x7fe3dc0aee80 2023-12-16T21:43:40.830 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:40.827+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe3dc0b0f40 con 0x7fe3dc0aee80 2023-12-16T21:43:41.495 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.491+0000 7fe3c3fff640 1 -- 172.21.15.46:0/410006926 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe3dc0b0f40 con 0x7fe3dc0aee80 2023-12-16T21:43:41.495 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2023-12-16T21:43:41.498 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3c80664a0 msgr2=0x7fe3c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3c80664a0 0x7fe3c8068960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0af000 tx=0x7fe3cc0a7000 comp rx=0 tx=0).stop 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0aee80 msgr2=0x7fe3dc142ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0aee80 0x7fe3dc142ae0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000c3e0 tx=0x7fe3d000c8a0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 shutdown_connections 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3c80664a0 0x7fe3c8068960 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc0b4bb0 0x7fe3dc071990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc0aee80 0x7fe3dc142ae0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 --2- 172.21.15.46:0/410006926 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0736b0 0x7fe3dc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 >> 172.21.15.46:0/410006926 conn(0x7fe3dc0b55e0 msgr2=0x7fe3dc09a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:41.499 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 shutdown_connections 2023-12-16T21:43:41.500 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.495+0000 7fe3e1b69640 1 -- 172.21.15.46:0/410006926 wait complete. 2023-12-16T21:43:41.529 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-16T21:43:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.651+0000 7fe3fbcf8640 1 Processor -- start 2023-12-16T21:43:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.651+0000 7fe3fbcf8640 1 -- start start 2023-12-16T21:43:41.658 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fbcf8640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fe3f40bc870 0x7fe3f40b4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fbcf8640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f4140ee0 con 0x7fe3f4140a90 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fbcf8640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3f40bcc50 con 0x7fe3f4149cd0 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fbcf8640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f40b4ff0 con 0x7fe3f40bc870 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fe3f4140a90 0x7fe3f4151060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46430/0 (socket says 172.21.15.46:46430) 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 -- 172.21.15.46:0/3723320174 learned_addr learned my addr 172.21.15.46:0/3723320174 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f9a94640 1 --2- 172.21.15.46:0/3723320174 >> v2:172.21.15.161:3300/0 conn(0x7fe3f40bc870 0x7fe3f40b4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899874803 0 0) 0x7fe3f4140ee0 con 0x7fe3f4140a90 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3dc003530 con 0x7fe3f4140a90 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2394912726 0 0) 0x7fe3dc003530 con 0x7fe3f4140a90 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 >> v2:172.21.15.161:3300/0 conn(0x7fe3f40bc870 msgr2=0x7fe3f40b4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 --2- 172.21.15.46:0/3723320174 >> v2:172.21.15.161:3300/0 conn(0x7fe3f40bc870 0x7fe3f40b4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 >> v1:172.21.15.161:6789/0 conn(0x7fe3f4149cd0 legacy=0x7fe3f4140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.659 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f40b3870 con 0x7fe3f4140a90 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7fe3e4001680 con 0x7fe3f4140a90 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 >> v1:172.21.15.46:6789/0 conn(0x7fe3f4140a90 legacy=0x7fe3f4151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc003980 0x7fe3dc003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0047d0 0x7fe3dc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc005190 0x7fe3dc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40b4ff0 con 0x7fe3dc005190 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc00c030 con 0x7fe3dc003980 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc00c1f0 con 0x7fe3dc0047d0 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc005190 0x7fe3dc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f9a94640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc003980 0x7fe3dc003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f9293640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0047d0 0x7fe3dc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 -- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0047d0 msgr2=0x7fe3dc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0047d0 0x7fe3dc004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 -- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc003980 msgr2=0x7fe3dc003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc003980 0x7fe3dc003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 -- 172.21.15.46:0/3723320174 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc003530 con 0x7fe3dc005190 2023-12-16T21:43:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f9293640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3dc0047d0 0x7fe3dc004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 -- 172.21.15.46:0/3723320174 reap_dead start 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3fa295640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc005190 0x7fe3dc00b990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c240 tx=0x7fe3e401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f842aa4de087d54 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e40a5b40 con 0x7fe3dc005190 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e40a5cc0 con 0x7fe3dc005190 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.655+0000 7fe3f8a92640 1 -- 172.21.15.46:0/3723320174 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4005b20 con 0x7fe3dc005190 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/3723320174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc005190 msgr2=0x7fe3dc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc005190 0x7fe3dc00b990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400c240 tx=0x7fe3e401cc40 comp rx=0 tx=0).stop 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/3723320174 shutdown_connections 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/3723320174 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3dc005190 0x7fe3dc00b990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/3723320174 >> 172.21.15.46:0/3723320174 conn(0x7fe3f4072390 msgr2=0x7fe3f40bbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/3723320174 shutdown_connections 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/3723320174 wait complete. 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 Processor -- start 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- start start 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc005190 0x7fe3f41df220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f40bb560 0x7fe3f40ba620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f40bab60 0x7fe3f4143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40ba190 con 0x7fe3f40bb560 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f4141ad0 con 0x7fe3dc005190 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40baff0 con 0x7fe3f40bab60 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f40bb560 0x7fe3f40ba620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f40bb560 0x7fe3f40ba620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35502/0 (socket says 172.21.15.46:35502) 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 -- 172.21.15.46:0/1333269571 learned_addr learned my addr 172.21.15.46:0/1333269571 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fa295640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f40bab60 0x7fe3f4143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9a94640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc005190 0x7fe3f41df220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 -- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f40bab60 msgr2=0x7fe3f4143760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f40bab60 0x7fe3f4143760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 -- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc005190 msgr2=0x7fe3f41df220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc005190 0x7fe3f41df220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 -- 172.21.15.46:0/1333269571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f4142020 con 0x7fe3f40bb560 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f9293640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f40bb560 0x7fe3f40ba620 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d5a0 tx=0x7fe3e800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f27fc640 1 -- 172.21.15.46:0/1333269571 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8005480 con 0x7fe3f40bb560 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f27fc640 1 -- 172.21.15.46:0/1333269571 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e8003620 con 0x7fe3f40bb560 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f41dfa70 con 0x7fe3f40bb560 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3f27fc640 1 -- 172.21.15.46:0/1333269571 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80039b0 con 0x7fe3f40bb560 2023-12-16T21:43:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.659+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f4141d40 con 0x7fe3f40bb560 2023-12-16T21:43:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.663+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f40ba340 con 0x7fe3f40bb560 2023-12-16T21:43:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.663+0000 7fe3f27fc640 1 -- 172.21.15.46:0/1333269571 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fe3e8003b10 con 0x7fe3f40bb560 2023-12-16T21:43:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.663+0000 7fe3f27fc640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3e00664a0 0x7fe3e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.663+0000 7fe3f27fc640 1 -- 172.21.15.46:0/1333269571 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe3e8083a10 con 0x7fe3f40bb560 2023-12-16T21:43:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.663+0000 7fe3f9a94640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3e00664a0 0x7fe3e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.668 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.663+0000 7fe3f9a94640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3e00664a0 0x7fe3e0068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec007380 tx=0x7fe3ec00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:41.670 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.667+0000 7fe3f27fc640 1 -- 172.21.15.46:0/1333269571 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7fe3f40ba340 con 0x7fe3f40bb560 2023-12-16T21:43:41.837 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.831+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe3f40ba340 con 0x7fe3f40bb560 2023-12-16T21:43:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.839+0000 7fe3f27fc640 1 -- 172.21.15.46:0/1333269571 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe3f40ba340 con 0x7fe3f40bb560 2023-12-16T21:43:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3e00664a0 msgr2=0x7fe3e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3e00664a0 0x7fe3e0068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec007380 tx=0x7fe3ec00b040 comp rx=0 tx=0).stop 2023-12-16T21:43:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f40bb560 msgr2=0x7fe3f40ba620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f40bb560 0x7fe3f40ba620 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800d5a0 tx=0x7fe3e800da60 comp rx=0 tx=0).stop 2023-12-16T21:43:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 shutdown_connections 2023-12-16T21:43:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fe3e00664a0 0x7fe3e0068960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fe3f40bab60 0x7fe3f4143760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f40bb560 0x7fe3f40ba620 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 --2- 172.21.15.46:0/1333269571 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fe3dc005190 0x7fe3f41df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 >> 172.21.15.46:0/1333269571 conn(0x7fe3f4072390 msgr2=0x7fe3f40b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 shutdown_connections 2023-12-16T21:43:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.843+0000 7fe3fbcf8640 1 -- 172.21.15.46:0/1333269571 wait complete. 2023-12-16T21:43:41.864 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2023-12-16T21:43:41.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7290df1640 1 Processor -- start 2023-12-16T21:43:41.981 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7290df1640 1 -- start start 2023-12-16T21:43:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7290df1640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f728c0b3940 0x7f728c0ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7290df1640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c0b2340 con 0x7f728c0b17a0 2023-12-16T21:43:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7290df1640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c0b27e0 con 0x7f728c0ae5b0 2023-12-16T21:43:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7290df1640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c0afe40 con 0x7f728c0b3940 2023-12-16T21:43:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7289d9b640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f728c0b3940 0x7f728c0ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f728ad9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f728c0b17a0 0x7f728c0b1c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46440/0 (socket says 172.21.15.46:46440) 2023-12-16T21:43:41.982 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f728ad9d640 1 -- 172.21.15.46:0/2389981358 learned_addr learned my addr 172.21.15.46:0/2389981358 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:41.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7289d9b640 1 -- 172.21.15.46:0/2389981358 >> v1:172.21.15.161:6789/0 conn(0x7f728c0ae5b0 legacy=0x7f728c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7289d9b640 1 -- 172.21.15.46:0/2389981358 >> v1:172.21.15.46:6789/0 conn(0x7f728c0b17a0 legacy=0x7f728c0b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f7289d9b640 1 -- 172.21.15.46:0/2389981358 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c0b2a30 con 0x7f728c0b3940 2023-12-16T21:43:41.983 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.979+0000 7f728959a640 1 -- 172.21.15.46:0/2389981358 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1272005584 0 0) 0x7f728c0b2340 con 0x7f728c0b17a0 2023-12-16T21:43:41.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7289d9b640 1 --2- 172.21.15.46:0/2389981358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0b3940 0x7f728c0ae990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7280009d90 tx=0x7f728001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7033d5e0f6d5f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:41.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728959a640 1 -- 172.21.15.46:0/2389981358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800a5950 con 0x7f728c0b3940 2023-12-16T21:43:41.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728959a640 1 -- 172.21.15.46:0/2389981358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72800a5ab0 con 0x7f728c0b3940 2023-12-16T21:43:41.985 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728959a640 1 -- 172.21.15.46:0/2389981358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72800a5e40 con 0x7f728c0b3940 2023-12-16T21:43:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- 172.21.15.46:0/2389981358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0b3940 msgr2=0x7f728c0ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 --2- 172.21.15.46:0/2389981358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0b3940 0x7f728c0ae990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7280009d90 tx=0x7f728001c890 comp rx=0 tx=0).stop 2023-12-16T21:43:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- 172.21.15.46:0/2389981358 shutdown_connections 2023-12-16T21:43:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 --2- 172.21.15.46:0/2389981358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0b3940 0x7f728c0ae990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- 172.21.15.46:0/2389981358 >> 172.21.15.46:0/2389981358 conn(0x7f728c0b48e0 msgr2=0x7f728c0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- 172.21.15.46:0/2389981358 shutdown_connections 2023-12-16T21:43:41.986 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- 172.21.15.46:0/2389981358 wait complete. 2023-12-16T21:43:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 Processor -- start 2023-12-16T21:43:41.987 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- start start 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0ae5b0 0x7f728c135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f728c0b17a0 0x7f728c1debe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728c0b3940 0x7f728c1df120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c13c9d0 con 0x7f728c0ae5b0 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c142ca0 con 0x7f728c0b3940 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7290df1640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c1e2690 con 0x7f728c0b17a0 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0ae5b0 0x7f728c135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728a59c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0ae5b0 0x7f728c135b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35514/0 (socket says 172.21.15.46:35514) 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728a59c640 1 -- 172.21.15.46:0/3211905037 learned_addr learned my addr 172.21.15.46:0/3211905037 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f7289d9b640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f728c0b17a0 0x7f728c1debe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.990 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728ad9d640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728c0b3940 0x7f728c1df120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728a59c640 1 -- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f728c0b17a0 msgr2=0x7f728c1debe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728a59c640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f728c0b17a0 0x7f728c1debe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728a59c640 1 -- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728c0b3940 msgr2=0x7f728c1df120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.983+0000 7f728a59c640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728c0b3940 0x7f728c1df120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f728a59c640 1 -- 172.21.15.46:0/3211905037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280009a70 con 0x7f728c0ae5b0 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f728a59c640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0ae5b0 0x7f728c135b20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f727c00c500 tx=0x7f727c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f727affd640 1 -- 172.21.15.46:0/3211905037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c005220 con 0x7f728c0ae5b0 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f727affd640 1 -- 172.21.15.46:0/3211905037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727c005380 con 0x7f728c0ae5b0 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c1e2970 con 0x7f728c0ae5b0 2023-12-16T21:43:41.991 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f727affd640 1 -- 172.21.15.46:0/3211905037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727c007270 con 0x7f728c0ae5b0 2023-12-16T21:43:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c1e2380 con 0x7f728c0ae5b0 2023-12-16T21:43:41.992 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c0679e0 con 0x7f728c0ae5b0 2023-12-16T21:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f727affd640 1 -- 172.21.15.46:0/3211905037 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f727c0073d0 con 0x7f728c0ae5b0 2023-12-16T21:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f727affd640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f726c0664a0 0x7f726c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f727affd640 1 -- 172.21.15.46:0/3211905037 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f727c082e10 con 0x7f728c0ae5b0 2023-12-16T21:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f7289d9b640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f726c0664a0 0x7f726c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:41.993 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.987+0000 7f7289d9b640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f726c0664a0 0x7f726c068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f72800af000 tx=0x7f72800a7070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:41.996 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:41.991+0000 7f727affd640 1 -- 172.21.15.46:0/3211905037 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f727c050650 con 0x7f728c0ae5b0 2023-12-16T21:43:42.208 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.203+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) v1 -- 0x7f728c0688f0 con 0x7f728c0ae5b0 2023-12-16T21:43:42.517 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f727affd640 1 -- 172.21.15.46:0/3211905037 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v82) v1 ==== 134+0+0 (secure 0 0 0) 0x7f727c053c70 con 0x7f728c0ae5b0 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f726c0664a0 msgr2=0x7f726c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f726c0664a0 0x7f726c068960 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f72800af000 tx=0x7f72800a7070 comp rx=0 tx=0).stop 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0ae5b0 msgr2=0x7f728c135b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0ae5b0 0x7f728c135b20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f727c00c500 tx=0x7f727c00c9c0 comp rx=0 tx=0).stop 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 shutdown_connections 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f726c0664a0 0x7f726c068960 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f728c0b3940 0x7f728c1df120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:42.520 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f728c0b17a0 0x7f728c1debe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 --2- 172.21.15.46:0/3211905037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f728c0ae5b0 0x7f728c135b20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 >> 172.21.15.46:0/3211905037 conn(0x7f728c0b48e0 msgr2=0x7f728c066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 shutdown_connections 2023-12-16T21:43:42.521 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:42.515+0000 7f7290df1640 1 -- 172.21.15.46:0/3211905037 wait complete. 2023-12-16T21:43:42.544 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2023-12-16T21:43:42.544 INFO:tasks.cephfs_test_runner: 2023-12-16T21:43:42.544 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-16T21:43:42.544 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2023-12-16T21:43:42.544 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2023-12-16T21:43:42.544 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-16T21:43:42.544 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: r.wait() 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 127: 'cephfs-shell -b /tmp/test-cephfsynu_kybc' 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner: 2023-12-16T21:43:42.545 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner:Ran 1 test in 42.407s 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner: 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner: 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner:====================================================================== 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2023-12-16T21:43:42.546 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: r.wait() 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 127: 'cephfs-shell -b /tmp/test-cephfsynu_kybc' 2023-12-16T21:43:42.547 INFO:tasks.cephfs_test_runner: 2023-12-16T21:43:42.547 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2023-12-16T21:43:42.621 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=0b95a3a1adfe46e488c13d683036431b Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph-c_c7586e95519b7be76f3b6e3a4b333888baf46bc9/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2023-12-16T21:43:42.623 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-16T21:43:42.632 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-12-16T21:43:42.641 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-12-16T21:43:42.641 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2023-12-16T21:43:42.641 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-16T21:43:42.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339656k,nr_inodes=4084914,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17020 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275312k,nr_inodes=818828,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275316k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2023-12-16T21:43:42.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:42.656 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2023-12-16T21:43:42.702 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-16T21:43:42.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:42.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-16T21:43:42.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-16T21:43:42.759 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2023-12-16T21:43:42.967 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-16T21:43:42.976 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-16T21:43:42.977 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.103+0000 7f8ae2d70640 1 Processor -- start 2023-12-16T21:43:43.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.103+0000 7f8ae2d70640 1 -- start start 2023-12-16T21:43:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae2d70640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc06c920 0x7f8adc0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae2d70640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8adc09dd50 0x7f8adc13ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae2d70640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8adc0af2e0 con 0x7f8adc0b3fd0 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae2d70640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8adc0b1f50 con 0x7f8adc09dd50 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae2d70640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc06cd00 con 0x7f8adc06c920 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8adc09dd50 0x7f8adc13ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f8adc09dd50 0x7f8adc13ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35524/0 (socket says 172.21.15.46:35524) 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8adbfff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8adc0b3fd0 0x7f8adc13e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46452/0 (socket says 172.21.15.46:46452) 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 -- 172.21.15.46:0/572097737 learned_addr learned my addr 172.21.15.46:0/572097737 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae0b0c640 1 --2- 172.21.15.46:0/572097737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc06c920 0x7f8adc0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 -- 172.21.15.46:0/572097737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc06c920 msgr2=0x7f8adc0afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 --2- 172.21.15.46:0/572097737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc06c920 0x7f8adc0afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 -- 172.21.15.46:0/572097737 >> v1:172.21.15.46:6789/0 conn(0x7f8adc0b3fd0 legacy=0x7f8adc13e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 -- 172.21.15.46:0/572097737 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8adc0af530 con 0x7f8adc09dd50 2023-12-16T21:43:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8adb7fe640 1 -- 172.21.15.46:0/572097737 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943669838 0 0) 0x7f8adc0af2e0 con 0x7f8adc0b3fd0 2023-12-16T21:43:43.112 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae0b0c640 1 --2- 172.21.15.46:0/572097737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc06c920 0x7f8adc0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8ae130d640 1 --2- 172.21.15.46:0/572097737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc09dd50 0x7f8adc13ea30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009e50 tx=0x7f8ad401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc364581ed450ff3 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8adb7fe640 1 -- 172.21.15.46:0/572097737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40a5640 con 0x7f8adc09dd50 2023-12-16T21:43:43.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8adb7fe640 1 -- 172.21.15.46:0/572097737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad40a57a0 con 0x7f8adc09dd50 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.107+0000 7f8adb7fe640 1 -- 172.21.15.46:0/572097737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40a5b30 con 0x7f8adc09dd50 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/572097737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc09dd50 msgr2=0x7f8adc13ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/572097737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc09dd50 0x7f8adc13ea30 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8ad4009e50 tx=0x7f8ad401c980 comp rx=0 tx=0).stop 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/572097737 shutdown_connections 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/572097737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc09dd50 0x7f8adc13ea30 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/572097737 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc06c920 0x7f8adc0afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/572097737 >> 172.21.15.46:0/572097737 conn(0x7f8adc0b48e0 msgr2=0x7f8adc0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/572097737 shutdown_connections 2023-12-16T21:43:43.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/572097737 wait complete. 2023-12-16T21:43:43.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 Processor -- start 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- start start 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8adc06c920 0x7f8adc13cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc09dd50 0x7f8adc06e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc1e0cc0 0x7f8adc1406d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc071b40 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc06fec0 con 0x7f8adc06c920 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8adc1e01e0 con 0x7f8adc09dd50 2023-12-16T21:43:43.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc1e0cc0 0x7f8adc1406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc1e0cc0 0x7f8adc1406d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35534/0 (socket says 172.21.15.46:35534) 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 -- 172.21.15.46:0/2256875340 learned_addr learned my addr 172.21.15.46:0/2256875340 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae0b0c640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8adc06c920 0x7f8adc13cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8adbfff640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc09dd50 0x7f8adc06e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 -- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc09dd50 msgr2=0x7f8adc06e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc09dd50 0x7f8adc06e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 -- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8adc06c920 msgr2=0x7f8adc13cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8adc06c920 0x7f8adc13cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.118 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 -- 172.21.15.46:0/2256875340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4009b30 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae0b0c640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8adc06c920 0x7f8adc13cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8adbfff640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc09dd50 0x7f8adc06e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae130d640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc1e0cc0 0x7f8adc1406d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40af040 tx=0x7f8ad4019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ad97fa640 1 -- 172.21.15.46:0/2256875340 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40b8880 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ad97fa640 1 -- 172.21.15.46:0/2256875340 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad40b89e0 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8adc1e04c0 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ad97fa640 1 -- 172.21.15.46:0/2256875340 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad40b8d90 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8adc13d490 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.111+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc0679e0 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.115+0000 7f8ad97fa640 1 -- 172.21.15.46:0/2256875340 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8ad40ca430 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.120 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.115+0000 7f8ad97fa640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8abc066470 0x7f8abc068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.115+0000 7f8ad97fa640 1 -- 172.21.15.46:0/2256875340 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8ad411f720 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.115+0000 7f8ae0b0c640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8abc066470 0x7f8abc068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.121 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.115+0000 7f8ae0b0c640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8abc066470 0x7f8abc068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8adc09dcc0 tx=0x7f8acc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.122 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.119+0000 7f8ad97fa640 1 -- 172.21.15.46:0/2256875340 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8ad40ed030 con 0x7f8adc1e0cc0 2023-12-16T21:43:43.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.271+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8adc067e50 con 0x7f8abc066470 2023-12-16T21:43:43.281 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ad97fa640 1 -- 172.21.15.46:0/2256875340 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49067 (secure 0 0 0) 0x7f8adc067e50 con 0x7f8abc066470 2023-12-16T21:43:43.282 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:43.282 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:43.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8abc066470 msgr2=0x7f8abc068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8abc066470 0x7f8abc068930 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8adc09dcc0 tx=0x7f8acc008040 comp rx=0 tx=0).stop 2023-12-16T21:43:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc1e0cc0 msgr2=0x7f8adc1406d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc1e0cc0 0x7f8adc1406d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8ad40af040 tx=0x7f8ad4019720 comp rx=0 tx=0).stop 2023-12-16T21:43:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 shutdown_connections 2023-12-16T21:43:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8abc066470 0x7f8abc068930 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.284 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8adc1e0cc0 0x7f8adc1406d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8adc09dd50 0x7f8adc06e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 --2- 172.21.15.46:0/2256875340 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8adc06c920 0x7f8adc13cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 >> 172.21.15.46:0/2256875340 conn(0x7f8adc0b48e0 msgr2=0x7f8adc0b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:43.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 shutdown_connections 2023-12-16T21:43:43.285 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.279+0000 7f8ae2d70640 1 -- 172.21.15.46:0/2256875340 wait complete. 2023-12-16T21:43:43.300 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-12-16T21:43:42.371305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236672,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738048,"statfs":{"total":773094113280,"available":772851761152,"internally_reserved":0,"allocated":3956736,"data_stored":2231782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10589,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-48,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001744"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-16T21:43:20.286524+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:20.286524+0000","last_peered":"2023-12-16T21:43:20.286524+0000","last_clean":"2023-12-16T21:43:20.286524+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:20.286524+0000","last_undegraded":"2023-12-16T21:43:20.286524+0000","last_fullsized":"2023-12-16T21:43:20.286524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":719117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.024},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.872}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90900000000000003}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.103},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.749}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.216}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.276},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":1.073},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.1419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":129275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.026}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.0669999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76900000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.1339999999999999}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.0980000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":129331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.89300000000000002}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.628},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":0.86699999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.94399999999999995}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.1819999999999999}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":128763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":0.89500000000000002}]},{"osd":5,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.5720000000000001,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":1.016}]},{"osd":7,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":0.94899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":360448,"data_stored":138770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.53900000000000003}]},{"osd":4,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.88700000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.84099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":339968,"data_stored":128729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.93400000000000005}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.871}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":360448,"data_stored":138770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47399999999999998}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0129999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.24},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.151}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.048}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.079}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":929792,"data_stored":719027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.872}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":1.121},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":0.95999999999999996}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":1.0249999999999999}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.077},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.91800000000000004}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.343},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":3.1440000000000001,"5min":3.1440000000000001,"15min":3.1440000000000001},"last":1.536}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:43.302 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:43.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25ab8f6640 1 Processor -- start 2023-12-16T21:43:43.417 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25ab8f6640 1 -- start start 2023-12-16T21:43:43.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25ab8f6640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25a40b4120 0x7f25a4147a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25ab8f6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 0x7f25a414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25ab8f6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a40b5ec0 con 0x7f25a40b70f0 2023-12-16T21:43:43.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25ab8f6640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a40b1df0 con 0x7f25a40b4120 2023-12-16T21:43:43.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25ab8f6640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4148460 con 0x7f25a4147fd0 2023-12-16T21:43:43.420 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9692640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f25a40b70f0 0x7f25a40b3a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46454/0 (socket says 172.21.15.46:46454) 2023-12-16T21:43:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9692640 1 -- 172.21.15.46:0/3541033160 learned_addr learned my addr 172.21.15.46:0/3541033160 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199057106 0 0) 0x7f25a40b5ec0 con 0x7f25a40b70f0 2023-12-16T21:43:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a8e91640 1 --2- 172.21.15.46:0/3541033160 >> v2:172.21.15.161:3300/0 conn(0x7f25a40b4120 0x7f25a4147a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9e93640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 0x7f25a414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.421 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2590003530 con 0x7f25a40b70f0 2023-12-16T21:43:43.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 153876046 0 0) 0x7f2590003530 con 0x7f25a40b70f0 2023-12-16T21:43:43.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 msgr2=0x7f25a414ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:43.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 0x7f25a414ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 >> v2:172.21.15.161:3300/0 conn(0x7f25a40b4120 msgr2=0x7f25a4147a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:43.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 --2- 172.21.15.46:0/3541033160 >> v2:172.21.15.161:3300/0 conn(0x7f25a40b4120 0x7f25a4147a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a40b74d0 con 0x7f25a40b70f0 2023-12-16T21:43:43.422 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f25a0001680 con 0x7f25a40b70f0 2023-12-16T21:43:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 >> v1:172.21.15.46:6789/0 conn(0x7f25a40b70f0 legacy=0x7f25a40b3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590003980 0x7f2590003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 0x7f25900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2590004d10 0x7f2590005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40b1df0 con 0x7f2590004d10 2023-12-16T21:43:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4148460 con 0x7f2590003980 2023-12-16T21:43:43.423 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f259000b9c0 con 0x7f25a4147fd0 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9e93640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2590004d10 0x7f2590005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9692640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590003980 0x7f2590003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a8e91640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 0x7f25900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9e93640 1 -- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 msgr2=0x7f25900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9e93640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a4147fd0 0x7f25900047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9e93640 1 -- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590003980 msgr2=0x7f2590003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9e93640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590003980 0x7f2590003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.415+0000 7f25a9e93640 1 -- 172.21.15.46:0/3541033160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590003530 con 0x7f2590004d10 2023-12-16T21:43:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25a9e93640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2590004d10 0x7f2590005190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f259c009fc0 tx=0x7f259c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8228ef16922ff10f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0a5c40 con 0x7f2590004d10 2023-12-16T21:43:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259c00f040 con 0x7f2590004d10 2023-12-16T21:43:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f259bfff640 1 -- 172.21.15.46:0/3541033160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259c0ad440 con 0x7f2590004d10 2023-12-16T21:43:43.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- 172.21.15.46:0/3541033160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2590004d10 msgr2=0x7f2590005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 --2- 172.21.15.46:0/3541033160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2590004d10 0x7f2590005190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f259c009fc0 tx=0x7f259c01cc30 comp rx=0 tx=0).stop 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25a9e93640 1 -- 172.21.15.46:0/3541033160 reap_dead start 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- 172.21.15.46:0/3541033160 shutdown_connections 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- 172.21.15.46:0/3541033160 >> 172.21.15.46:0/3541033160 conn(0x7f25a406a980 msgr2=0x7f25a406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- 172.21.15.46:0/3541033160 shutdown_connections 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- 172.21.15.46:0/3541033160 wait complete. 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 Processor -- start 2023-12-16T21:43:43.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- start start 2023-12-16T21:43:43.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590004d10 0x7f25a40b8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a40ba9c0 0x7f25a41d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a41da450 0x7f25a41dce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41da8e0 con 0x7f25a40ba9c0 2023-12-16T21:43:43.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41d9ac0 con 0x7f2590004d10 2023-12-16T21:43:43.430 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25ab8f6640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40b9550 con 0x7f25a41da450 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25a8e91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a40ba9c0 0x7f25a41d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25a8e91640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a40ba9c0 0x7f25a41d9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35550/0 (socket says 172.21.15.46:35550) 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.419+0000 7f25a8e91640 1 -- 172.21.15.46:0/1070049617 learned_addr learned my addr 172.21.15.46:0/1070049617 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a9e93640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a41da450 0x7f25a41dce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a9692640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590004d10 0x7f25a40b8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a8e91640 1 -- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a41da450 msgr2=0x7f25a41dce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a8e91640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a41da450 0x7f25a41dce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a8e91640 1 -- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590004d10 msgr2=0x7f25a40b8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.431 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a8e91640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590004d10 0x7f25a40b8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a8e91640 1 -- 172.21.15.46:0/1070049617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a413f320 con 0x7f25a40ba9c0 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a8e91640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a40ba9c0 0x7f25a41d9f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f25940116a0 tx=0x7f2594011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f2599ffb640 1 -- 172.21.15.46:0/1070049617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400f9c0 con 0x7f25a40ba9c0 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f2599ffb640 1 -- 172.21.15.46:0/1070049617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259400fb20 con 0x7f25a40ba9c0 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a413ef20 con 0x7f25a40ba9c0 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f2599ffb640 1 -- 172.21.15.46:0/1070049617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259400feb0 con 0x7f25a40ba9c0 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a40b8f50 con 0x7f25a40ba9c0 2023-12-16T21:43:43.432 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a40679e0 con 0x7f25a40ba9c0 2023-12-16T21:43:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f2599ffb640 1 -- 172.21.15.46:0/1070049617 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f2594015460 con 0x7f25a40ba9c0 2023-12-16T21:43:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f2599ffb640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2588066470 0x7f2588068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f2599ffb640 1 -- 172.21.15.46:0/1070049617 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f25940843f0 con 0x7f25a40ba9c0 2023-12-16T21:43:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a9692640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2588066470 0x7f2588068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.433 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.423+0000 7f25a9692640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2588066470 0x7f2588068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f25a0001060 tx=0x7f25a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.434 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.427+0000 7f2599ffb640 1 -- 172.21.15.46:0/1070049617 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f2594051c00 con 0x7f25a40ba9c0 2023-12-16T21:43:43.585 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.583+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25a413dfc0 con 0x7f2588066470 2023-12-16T21:43:43.591 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.587+0000 7f2599ffb640 1 -- 172.21.15.46:0/1070049617 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49067 (secure 0 0 0) 0x7f25a413dfc0 con 0x7f2588066470 2023-12-16T21:43:43.592 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:43.592 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:43.593 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2588066470 msgr2=0x7f2588068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2588066470 0x7f2588068930 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f25a0001060 tx=0x7f25a0009040 comp rx=0 tx=0).stop 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a40ba9c0 msgr2=0x7f25a41d9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a40ba9c0 0x7f25a41d9f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f25940116a0 tx=0x7f2594011b60 comp rx=0 tx=0).stop 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 shutdown_connections 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2588066470 0x7f2588068930 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25a41da450 0x7f25a41dce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25a40ba9c0 0x7f25a41d9f10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.594 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 --2- 172.21.15.46:0/1070049617 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f2590004d10 0x7f25a40b8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 >> 172.21.15.46:0/1070049617 conn(0x7f25a406a980 msgr2=0x7f25a413d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 shutdown_connections 2023-12-16T21:43:43.595 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.591+0000 7f25ab8f6640 1 -- 172.21.15.46:0/1070049617 wait complete. 2023-12-16T21:43:43.611 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-12-16T21:43:42.371305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236672,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738048,"statfs":{"total":773094113280,"available":772851761152,"internally_reserved":0,"allocated":3956736,"data_stored":2231782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10589,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-48,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001744"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-16T21:43:20.286524+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:20.286524+0000","last_peered":"2023-12-16T21:43:20.286524+0000","last_clean":"2023-12-16T21:43:20.286524+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:20.286524+0000","last_undegraded":"2023-12-16T21:43:20.286524+0000","last_fullsized":"2023-12-16T21:43:20.286524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":719117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.024},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.872}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90900000000000003}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.103},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.749}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.216}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.276},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":1.073},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.1419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":129275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.026}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.0669999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76900000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.1339999999999999}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.0980000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":129331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.89300000000000002}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.628},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":0.86699999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.94399999999999995}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.1819999999999999}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":128763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":0.89500000000000002}]},{"osd":5,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.5720000000000001,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":1.016}]},{"osd":7,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":0.94899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":360448,"data_stored":138770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.53900000000000003}]},{"osd":4,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.88700000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.84099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":339968,"data_stored":128729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.93400000000000005}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.871}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":360448,"data_stored":138770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47399999999999998}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0129999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.24},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.151}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.048}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.079}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":929792,"data_stored":719027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.872}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":1.121},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":0.95999999999999996}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":1.0249999999999999}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.077},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.91800000000000004}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.343},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":3.1440000000000001,"5min":3.1440000000000001,"15min":3.1440000000000001},"last":1.536}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:43.612 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-16T21:43:43.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:43.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.743+0000 7febd97d2640 1 Processor -- start 2023-12-16T21:43:43.748 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.743+0000 7febd97d2640 1 -- start start 2023-12-16T21:43:43.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.743+0000 7febd97d2640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febd40af2e0 0x7febd4147080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.749 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.743+0000 7febd97d2640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7febd40ad7c0 0x7febd41475c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.743+0000 7febd97d2640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd40af770 con 0x7febd406c920 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.743+0000 7febd97d2640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40afc20 con 0x7febd40af2e0 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.743+0000 7febd97d2640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd40aeea0 con 0x7febd40ad7c0 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febd40af2e0 0x7febd4147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7febd40af2e0 0x7febd4147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35552/0 (socket says 172.21.15.46:35552) 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 -- 172.21.15.46:0/484849007 learned_addr learned my addr 172.21.15.46:0/484849007 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd37fe640 1 --2- 172.21.15.46:0/484849007 >> v2:172.21.15.161:3300/0 conn(0x7febd40ad7c0 0x7febd41475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.750 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 -- 172.21.15.46:0/484849007 >> v2:172.21.15.161:3300/0 conn(0x7febd40ad7c0 msgr2=0x7febd41475c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 --2- 172.21.15.46:0/484849007 >> v2:172.21.15.161:3300/0 conn(0x7febd40ad7c0 0x7febd41475c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 -- 172.21.15.46:0/484849007 >> v1:172.21.15.161:6789/0 conn(0x7febd406c920 legacy=0x7febd40ad0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:43.751 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 -- 172.21.15.46:0/484849007 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4147b00 con 0x7febd40af2e0 2023-12-16T21:43:43.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd27fc640 1 --2- 172.21.15.46:0/484849007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af2e0 0x7febd4147080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7febd40af1b0 tx=0x7febc401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbb34e8d031eb0b server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd1ffb640 1 -- 172.21.15.46:0/484849007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc40ac600 con 0x7febd40af2e0 2023-12-16T21:43:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd1ffb640 1 -- 172.21.15.46:0/484849007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc40a8050 con 0x7febd40af2e0 2023-12-16T21:43:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd1ffb640 1 -- 172.21.15.46:0/484849007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc40ac980 con 0x7febd40af2e0 2023-12-16T21:43:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 -- 172.21.15.46:0/484849007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af2e0 msgr2=0x7febd4147080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 --2- 172.21.15.46:0/484849007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af2e0 0x7febd4147080 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7febd40af1b0 tx=0x7febc401c3a0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 -- 172.21.15.46:0/484849007 shutdown_connections 2023-12-16T21:43:43.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 --2- 172.21.15.46:0/484849007 >> v2:172.21.15.161:3300/0 conn(0x7febd40ad7c0 0x7febd41475c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 --2- 172.21.15.46:0/484849007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd40af2e0 0x7febd4147080 secure :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7febd40af1b0 tx=0x7febc401c3a0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 -- 172.21.15.46:0/484849007 >> 172.21.15.46:0/484849007 conn(0x7febd40b48e0 msgr2=0x7febd40b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 -- 172.21.15.46:0/484849007 shutdown_connections 2023-12-16T21:43:43.755 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.747+0000 7febd97d2640 1 -- 172.21.15.46:0/484849007 wait complete. 2023-12-16T21:43:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 Processor -- start 2023-12-16T21:43:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 -- start start 2023-12-16T21:43:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd406c920 0x7febd413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40ad7c0 0x7febd413f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413f6f0 0x7febd4141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd41ddaa0 con 0x7febd413f6f0 2023-12-16T21:43:43.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd409dbd0 con 0x7febd40ad7c0 2023-12-16T21:43:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd41dd420 con 0x7febd406c920 2023-12-16T21:43:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413f6f0 0x7febd4141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413f6f0 0x7febd4141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35554/0 (socket says 172.21.15.46:35554) 2023-12-16T21:43:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 -- 172.21.15.46:0/3519653290 learned_addr learned my addr 172.21.15.46:0/3519653290 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd27fc640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40ad7c0 0x7febd413f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd2ffd640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd406c920 0x7febd413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 -- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd406c920 msgr2=0x7febd413ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd406c920 0x7febd413ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 -- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40ad7c0 msgr2=0x7febd413f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40ad7c0 0x7febd413f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 -- 172.21.15.46:0/3519653290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc40094d0 con 0x7febd413f6f0 2023-12-16T21:43:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd37fe640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413f6f0 0x7febd4141ec0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7febcc007e60 tx=0x7febcc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febbb7fe640 1 -- 172.21.15.46:0/3519653290 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc014420 con 0x7febd413f6f0 2023-12-16T21:43:43.760 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febbb7fe640 1 -- 172.21.15.46:0/3519653290 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc0145a0 con 0x7febd413f6f0 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd41dd6c0 con 0x7febd413f6f0 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febbb7fe640 1 -- 172.21.15.46:0/3519653290 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc014950 con 0x7febd413f6f0 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd413fe90 con 0x7febd413f6f0 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd413bdc0 con 0x7febd413f6f0 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.751+0000 7febbb7fe640 1 -- 172.21.15.46:0/3519653290 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7febcc014ab0 con 0x7febd413f6f0 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.755+0000 7febbb7fe640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.755+0000 7febbb7fe640 1 -- 172.21.15.46:0/3519653290 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7febcc0844b0 con 0x7febd413f6f0 2023-12-16T21:43:43.761 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.755+0000 7febd2ffd640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.755+0000 7febd2ffd640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7febc8004e30 tx=0x7febc8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:43.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.759+0000 7febbb7fe640 1 -- 172.21.15.46:0/3519653290 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7febcc051cc0 con 0x7febd413f6f0 2023-12-16T21:43:43.912 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.907+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febd40679e0 con 0x7febc0066470 2023-12-16T21:43:43.918 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febbb7fe640 1 -- 172.21.15.46:0/3519653290 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49067 (secure 0 0 0) 0x7febd40679e0 con 0x7febc0066470 2023-12-16T21:43:43.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:43.919 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 msgr2=0x7febc0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7febc8004e30 tx=0x7febc8009040 comp rx=0 tx=0).stop 2023-12-16T21:43:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413f6f0 msgr2=0x7febd4141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:43.921 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413f6f0 0x7febd4141ec0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7febcc007e60 tx=0x7febcc00e4b0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 shutdown_connections 2023-12-16T21:43:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7febc0066470 0x7febc0068930 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd413f6f0 0x7febd4141ec0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7febd40ad7c0 0x7febd413f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 --2- 172.21.15.46:0/3519653290 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7febd406c920 0x7febd413ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 >> 172.21.15.46:0/3519653290 conn(0x7febd40b48e0 msgr2=0x7febd4139e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.915+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 shutdown_connections 2023-12-16T21:43:43.923 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:43.919+0000 7febd97d2640 1 -- 172.21.15.46:0/3519653290 wait complete. 2023-12-16T21:43:43.938 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-12-16T21:43:42.371305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236672,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738048,"statfs":{"total":773094113280,"available":772851761152,"internally_reserved":0,"allocated":3956736,"data_stored":2231782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10589,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-48,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001744"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-12-16T21:43:20.286524+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:20.286524+0000","last_peered":"2023-12-16T21:43:20.286524+0000","last_clean":"2023-12-16T21:43:20.286524+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:20.286524+0000","last_undegraded":"2023-12-16T21:43:20.286524+0000","last_fullsized":"2023-12-16T21:43:20.286524+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343900,"statfs":{"total":96636764160,"available":96608153600,"internally_reserved":0,"allocated":933888,"data_stored":719117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.024},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.872}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90900000000000003}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":1.103},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.749}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.216}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":1.1799999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.68200000000000005}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.276},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.97499999999999998}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":1.073},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.1419999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":129275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.026}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.96299999999999997},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.0669999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.76900000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.1859999999999999},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.72699999999999998}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.1339999999999999}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.0980000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":344064,"data_stored":129331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.89300000000000002}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.628},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.67200000000000004}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.76900000000000002}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":0.86699999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":0.94399999999999995}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.1819999999999999}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.97199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":344064,"data_stored":128763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.71699999999999997}]},{"osd":1,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.76000000000000001}]},{"osd":2,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.86599999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.53600000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":0.89500000000000002}]},{"osd":5,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.97199999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.64000000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.5720000000000001,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":1.016}]},{"osd":7,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":0.80200000000000005},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":0.94899999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":360448,"data_stored":138770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59099999999999997}]},{"osd":2,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.495},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.53900000000000003}]},{"osd":4,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.80000000000000004}]},{"osd":5,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.88700000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.76700000000000002}]},{"osd":7,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.84099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31516,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340324,"statfs":{"total":96636764160,"available":96604491776,"internally_reserved":0,"allocated":339968,"data_stored":128729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.52100000000000002}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.44400000000000001}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.40000000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.73399999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.93400000000000005}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.871}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31528,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340312,"statfs":{"total":96636764160,"available":96604479488,"internally_reserved":0,"allocated":360448,"data_stored":138770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.47399999999999998}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.55700000000000005}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.73399999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.0129999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":1.24},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.151}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.048}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.2150000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.079}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":929792,"data_stored":719027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.71199999999999997}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.872}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.56100000000000005}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":1.121},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":0.95999999999999996}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":1.0249999999999999}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.077},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.91800000000000004}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.343},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":3.1440000000000001,"5min":3.1440000000000001,"15min":3.1440000000000001},"last":1.536}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:43.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-16T21:43:44.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1773114640 1 Processor -- start 2023-12-16T21:43:44.065 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1773114640 1 -- start start 2023-12-16T21:43:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1773114640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c0ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1773114640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f176c06c6c0 0x7f176c0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1773114640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0b27e0 con 0x7f176c0ae5b0 2023-12-16T21:43:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1773114640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c0afe40 con 0x7f176c06c6c0 2023-12-16T21:43:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1773114640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b2a30 con 0x7f176c0b3940 2023-12-16T21:43:44.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1770eb0640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f176c0ae5b0 0x7f176c0b3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46468/0 (socket says 172.21.15.46:46468) 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f1770eb0640 1 -- 172.21.15.46:0/2303712520 learned_addr learned my addr 172.21.15.46:0/2303712520 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f17716b1640 1 --2- 172.21.15.46:0/2303712520 >> v2:172.21.15.46:3300/0 conn(0x7f176c06c6c0 0x7f176c0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f176bfff640 1 --2- 172.21.15.46:0/2303712520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c0ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f17716b1640 1 -- 172.21.15.46:0/2303712520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 msgr2=0x7f176c0ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f17716b1640 1 --2- 172.21.15.46:0/2303712520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c0ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f17716b1640 1 -- 172.21.15.46:0/2303712520 >> v1:172.21.15.46:6789/0 conn(0x7f176c0ae5b0 legacy=0x7f176c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f17716b1640 1 -- 172.21.15.46:0/2303712520 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0b2d10 con 0x7f176c06c6c0 2023-12-16T21:43:44.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.063+0000 7f176b7fe640 1 -- 172.21.15.46:0/2303712520 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488081445 0 0) 0x7f176c0b27e0 con 0x7f176c0ae5b0 2023-12-16T21:43:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f17716b1640 1 --2- 172.21.15.46:0/2303712520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b17a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1764009d90 tx=0x7f176401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1053d3999db8603 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f176b7fe640 1 -- 172.21.15.46:0/2303712520 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17640a5870 con 0x7f176c06c6c0 2023-12-16T21:43:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f176b7fe640 1 -- 172.21.15.46:0/2303712520 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17640a59d0 con 0x7f176c06c6c0 2023-12-16T21:43:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f176b7fe640 1 -- 172.21.15.46:0/2303712520 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17640a5d60 con 0x7f176c06c6c0 2023-12-16T21:43:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- 172.21.15.46:0/2303712520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 msgr2=0x7f176c0b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:44.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 --2- 172.21.15.46:0/2303712520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b17a0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1764009d90 tx=0x7f176401c890 comp rx=0 tx=0).stop 2023-12-16T21:43:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- 172.21.15.46:0/2303712520 shutdown_connections 2023-12-16T21:43:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 --2- 172.21.15.46:0/2303712520 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b17a0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 --2- 172.21.15.46:0/2303712520 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c0ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- 172.21.15.46:0/2303712520 >> 172.21.15.46:0/2303712520 conn(0x7f176c0b48e0 msgr2=0x7f176c0b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- 172.21.15.46:0/2303712520 shutdown_connections 2023-12-16T21:43:44.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- 172.21.15.46:0/2303712520 wait complete. 2023-12-16T21:43:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 Processor -- start 2023-12-16T21:43:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- start start 2023-12-16T21:43:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f176c0ae5b0 0x7f176c1e03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c13f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:44.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c09cfe0 con 0x7f176c06c6c0 2023-12-16T21:43:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c09aeb0 con 0x7f176c0ae5b0 2023-12-16T21:43:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1773114640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c09d4a0 con 0x7f176c0b3940 2023-12-16T21:43:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35568/0 (socket says 172.21.15.46:35568) 2023-12-16T21:43:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 -- 172.21.15.46:0/876433795 learned_addr learned my addr 172.21.15.46:0/876433795 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:44.075 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f176bfff640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f176c0ae5b0 0x7f176c1e03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f17716b1640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c13f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 -- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 msgr2=0x7f176c13f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c13f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 -- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f176c0ae5b0 msgr2=0x7f176c1e03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f176c0ae5b0 0x7f176c1e03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 -- 172.21.15.46:0/876433795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764009a70 con 0x7f176c06c6c0 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.067+0000 7f1770eb0640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b6420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f17600011f0 tx=0x7f176000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:44.076 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f17697fa640 1 -- 172.21.15.46:0/876433795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17600074e0 con 0x7f176c06c6c0 2023-12-16T21:43:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f17697fa640 1 -- 172.21.15.46:0/876433795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1760007640 con 0x7f176c06c6c0 2023-12-16T21:43:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c09b320 con 0x7f176c06c6c0 2023-12-16T21:43:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f17697fa640 1 -- 172.21.15.46:0/876433795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1760004a10 con 0x7f176c06c6c0 2023-12-16T21:43:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c09a990 con 0x7f176c06c6c0 2023-12-16T21:43:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c0679e0 con 0x7f176c06c6c0 2023-12-16T21:43:44.077 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f17697fa640 1 -- 172.21.15.46:0/876433795 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f1760004b70 con 0x7f176c06c6c0 2023-12-16T21:43:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f17697fa640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1750066470 0x7f1750068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f17697fa640 1 -- 172.21.15.46:0/876433795 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1760083020 con 0x7f176c06c6c0 2023-12-16T21:43:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f176bfff640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1750066470 0x7f1750068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:44.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.071+0000 7f176bfff640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1750066470 0x7f1750068930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f176c0b6d00 tx=0x7f1758008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:44.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.075+0000 7f17697fa640 1 -- 172.21.15.46:0/876433795 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f17600508e0 con 0x7f176c06c6c0 2023-12-16T21:43:44.234 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.231+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f176c09b160 con 0x7f176c06c6c0 2023-12-16T21:43:44.235 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.231+0000 7f17697fa640 1 -- 172.21.15.46:0/876433795 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+12986 (secure 0 0 0) 0x7f1760018090 con 0x7f176c06c6c0 2023-12-16T21:43:44.235 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":33,"fsid":"52c183c6-9df9-436e-b409-f347731c6c1e","created":"2023-12-16T21:42:15.612492+0000","modified":"2023-12-16T21:43:41.490061+0000","last_up_change":"2023-12-16T21:42:27.188581+0000","last_in_change":"2023-12-16T21:42:18.641021+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-16T21:42:28.427811+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a6d39371-17a2-4ed3-90aa-eff61ee76362","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6801","nonce":1629025463}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6803","nonce":1629025463}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6807","nonce":1629025463}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1629025463},{"type":"v1","addr":"172.21.15.46:6805","nonce":1629025463}]},"public_addr":"172.21.15.46:6801/1629025463","cluster_addr":"172.21.15.46:6803/1629025463","heartbeat_back_addr":"172.21.15.46:6807/1629025463","heartbeat_front_addr":"172.21.15.46:6805/1629025463","state":["exists","up"]},{"osd":1,"uuid":"a0fd9f81-9b98-46c5-b4e6-82adbcfc1064","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6825","nonce":491328631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6827","nonce":491328631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6831","nonce":491328631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":491328631},{"type":"v1","addr":"172.21.15.46:6829","nonce":491328631}]},"public_addr":"172.21.15.46:6825/491328631","cluster_addr":"172.21.15.46:6827/491328631","heartbeat_back_addr":"172.21.15.46:6831/491328631","heartbeat_front_addr":"172.21.15.46:6829/491328631","state":["exists","up"]},{"osd":2,"uuid":"63203227-89d5-467b-a0ef-6e66270c12a2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6810","nonce":3526272141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6814","nonce":3526272141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6823","nonce":3526272141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":3526272141},{"type":"v1","addr":"172.21.15.46:6818","nonce":3526272141}]},"public_addr":"172.21.15.46:6810/3526272141","cluster_addr":"172.21.15.46:6814/3526272141","heartbeat_back_addr":"172.21.15.46:6823/3526272141","heartbeat_front_addr":"172.21.15.46:6818/3526272141","state":["exists","up"]},{"osd":3,"uuid":"8225ce9b-2382-468e-8aaf-2dc5c7807217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6809","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6811","nonce":2268558465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6813","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6815","nonce":2268558465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6821","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6822","nonce":2268558465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6817","nonce":2268558465},{"type":"v1","addr":"172.21.15.46:6819","nonce":2268558465}]},"public_addr":"172.21.15.46:6811/2268558465","cluster_addr":"172.21.15.46:6815/2268558465","heartbeat_back_addr":"172.21.15.46:6822/2268558465","heartbeat_front_addr":"172.21.15.46:6819/2268558465","state":["exists","up"]},{"osd":4,"uuid":"1c1d186f-c49d-4a50-986f-c4a20dee5f0d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6800","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6801","nonce":3753525888}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6802","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6803","nonce":3753525888}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6806","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6807","nonce":3753525888}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6804","nonce":3753525888},{"type":"v1","addr":"172.21.15.161:6805","nonce":3753525888}]},"public_addr":"172.21.15.161:6801/3753525888","cluster_addr":"172.21.15.161:6803/3753525888","heartbeat_back_addr":"172.21.15.161:6807/3753525888","heartbeat_front_addr":"172.21.15.161:6805/3753525888","state":["exists","up"]},{"osd":5,"uuid":"a7e2dcfb-a4c2-4e55-90a4-79e4590292b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6808","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6809","nonce":1602903861}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6810","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6811","nonce":1602903861}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6814","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6815","nonce":1602903861}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6812","nonce":1602903861},{"type":"v1","addr":"172.21.15.161:6813","nonce":1602903861}]},"public_addr":"172.21.15.161:6809/1602903861","cluster_addr":"172.21.15.161:6811/1602903861","heartbeat_back_addr":"172.21.15.161:6815/1602903861","heartbeat_front_addr":"172.21.15.161:6813/1602903861","state":["exists","up"]},{"osd":6,"uuid":"0e507728-c7c8-4fe2-98c1-47c2315463a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6824","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6825","nonce":2291774553}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6826","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6827","nonce":2291774553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6830","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6831","nonce":2291774553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6828","nonce":2291774553},{"type":"v1","addr":"172.21.15.161:6829","nonce":2291774553}]},"public_addr":"172.21.15.161:6825/2291774553","cluster_addr":"172.21.15.161:6827/2291774553","heartbeat_back_addr":"172.21.15.161:6831/2291774553","heartbeat_front_addr":"172.21.15.161:6829/2291774553","state":["exists","up"]},{"osd":7,"uuid":"bccb177c-3eb8-44e6-9694-0b3c11292cff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6816","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6817","nonce":2824550480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6818","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6819","nonce":2824550480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6822","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6823","nonce":2824550480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6820","nonce":2824550480},{"type":"v1","addr":"172.21.15.161:6821","nonce":2824550480}]},"public_addr":"172.21.15.161:6817/2824550480","cluster_addr":"172.21.15.161:6819/2824550480","heartbeat_back_addr":"172.21.15.161:6823/2824550480","heartbeat_front_addr":"172.21.15.161:6821/2824550480","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.202766+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.283067+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.272707+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.325852+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.129140+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.127136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.241645+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-16T21:42:24.142192+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6833/3724256427":"2023-12-17T21:43:38.280260+0000","172.21.15.46:6832/3724256427":"2023-12-17T21:43:38.280260+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-16T21:43:44.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1750066470 msgr2=0x7f1750068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:44.237 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1750066470 0x7f1750068930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f176c0b6d00 tx=0x7f1758008040 comp rx=0 tx=0).stop 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 msgr2=0x7f176c0b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b6420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f17600011f0 tx=0x7f176000c9b0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 shutdown_connections 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1750066470 0x7f1750068930 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f176c0b3940 0x7f176c13f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f176c0ae5b0 0x7f176c1e03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 --2- 172.21.15.46:0/876433795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f176c06c6c0 0x7f176c0b6420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 >> 172.21.15.46:0/876433795 conn(0x7f176c0b48e0 msgr2=0x7f176c0b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:44.238 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 shutdown_connections 2023-12-16T21:43:44.239 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:44.235+0000 7f1773114640 1 -- 172.21.15.46:0/876433795 wait complete. 2023-12-16T21:43:45.255 INFO:tasks.ceph:Scrubbing osd.0 2023-12-16T21:43:45.255 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 Processor -- start 2023-12-16T21:43:45.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 -- start start 2023-12-16T21:43:45.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f659006c920 0x7f65900ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900aebd0 0x7f65900b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65900b2cd0 0x7f65900b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f659006cd00 con 0x7f65900b2cd0 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65900ad230 con 0x7f659006c920 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6597df4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65900af4e0 con 0x7f65900aebd0 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65900b2cd0 0x7f65900b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f65900b2cd0 0x7f65900b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35576/0 (socket says 172.21.15.46:35576) 2023-12-16T21:43:45.401 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 -- 172.21.15.46:0/426208254 learned_addr learned my addr 172.21.15.46:0/426208254 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6595b90640 1 --2- 172.21.15.46:0/426208254 >> v2:172.21.15.161:3300/0 conn(0x7f659006c920 0x7f65900ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f659538f640 1 --2- 172.21.15.46:0/426208254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900aebd0 0x7f65900b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 -- 172.21.15.46:0/426208254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900aebd0 msgr2=0x7f65900b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 --2- 172.21.15.46:0/426208254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900aebd0 0x7f65900b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 -- 172.21.15.46:0/426208254 >> v2:172.21.15.161:3300/0 conn(0x7f659006c920 msgr2=0x7f65900ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 --2- 172.21.15.46:0/426208254 >> v2:172.21.15.161:3300/0 conn(0x7f659006c920 0x7f65900ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.395+0000 7f6596391640 1 -- 172.21.15.46:0/426208254 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65900acf50 con 0x7f65900b2cd0 2023-12-16T21:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6596391640 1 --2- 172.21.15.46:0/426208254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900b2cd0 0x7f65900b3160 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f65900b39b0 tx=0x7f658001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2757e2795c1565 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6594b8e640 1 -- 172.21.15.46:0/426208254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65800a5790 con 0x7f65900b2cd0 2023-12-16T21:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6594b8e640 1 -- 172.21.15.46:0/426208254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65800a58f0 con 0x7f65900b2cd0 2023-12-16T21:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6594b8e640 1 -- 172.21.15.46:0/426208254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65800a5c80 con 0x7f65900b2cd0 2023-12-16T21:43:45.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 -- 172.21.15.46:0/426208254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900b2cd0 msgr2=0x7f65900b3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 --2- 172.21.15.46:0/426208254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900b2cd0 0x7f65900b3160 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f65900b39b0 tx=0x7f658001c910 comp rx=0 tx=0).stop 2023-12-16T21:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 -- 172.21.15.46:0/426208254 shutdown_connections 2023-12-16T21:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 --2- 172.21.15.46:0/426208254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900b2cd0 0x7f65900b3160 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 --2- 172.21.15.46:0/426208254 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900aebd0 0x7f65900b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 --2- 172.21.15.46:0/426208254 >> v2:172.21.15.161:3300/0 conn(0x7f659006c920 0x7f65900ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 -- 172.21.15.46:0/426208254 >> 172.21.15.46:0/426208254 conn(0x7f65900b4890 msgr2=0x7f65900b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:45.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 -- 172.21.15.46:0/426208254 shutdown_connections 2023-12-16T21:43:45.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.399+0000 7f6597df4640 1 -- 172.21.15.46:0/426208254 wait complete. 2023-12-16T21:43:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 Processor -- start 2023-12-16T21:43:45.409 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 -- start start 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659006c920 0x7f65901e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900aebd0 0x7f659014c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900b2cd0 0x7f6590143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901e3290 con 0x7f65900aebd0 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901e3570 con 0x7f659006c920 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65901e3850 con 0x7f65900b2cd0 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900aebd0 0x7f659014c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.410 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900aebd0 0x7f659014c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35578/0 (socket says 172.21.15.46:35578) 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 -- 172.21.15.46:0/257418947 learned_addr learned my addr 172.21.15.46:0/257418947 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6596391640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900b2cd0 0x7f6590143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6595b90640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659006c920 0x7f65901e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 -- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900b2cd0 msgr2=0x7f6590143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f65900b2cd0 0x7f6590143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 -- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659006c920 msgr2=0x7f65901e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f659006c920 0x7f65901e2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.411 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 -- 172.21.15.46:0/257418947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580009b30 con 0x7f65900aebd0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f659538f640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900aebd0 0x7f659014c610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f658400d7d0 tx=0x7f658400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658400b6c0 con 0x7f65900aebd0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f658400b820 con 0x7f65900aebd0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659009b210 con 0x7f65900aebd0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658400bbd0 con 0x7f65900aebd0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659014cb50 con 0x7f65900aebd0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65901e36e0 con 0x7f65900aebd0 2023-12-16T21:43:45.412 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f658400bd30 con 0x7f65900aebd0 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.403+0000 7f658e7fc640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f657c0664a0 0x7f657c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6584084a10 con 0x7f65900aebd0 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f658e7fc640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7f657c06c300 0x7f657c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 --> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f657c06edd0 con 0x7f657c06c300 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6584084e60 con 0x7f65900aebd0 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f6595b90640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f657c0664a0 0x7f657c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f6596391640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7f657c06c300 0x7f657c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.413 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f6596391640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7f657c06c300 0x7f657c06e720 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f6595b90640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f657c0664a0 0x7f657c068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f658800b9d0 tx=0x7f6588008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.414 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.407+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== osd.0 v2:172.21.15.46:6800/1629025463 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f657c06edd0 con 0x7f657c06c300 2023-12-16T21:43:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.419+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 --> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f659009de50 con 0x7f657c06c300 2023-12-16T21:43:45.424 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.419+0000 7f658e7fc640 1 -- 172.21.15.46:0/257418947 <== osd.0 v2:172.21.15.46:6800/1629025463 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f65800090d0 con 0x7f657c06c300 2023-12-16T21:43:45.425 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:45.425 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:45.425 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:45.426 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.419+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7f657c06c300 msgr2=0x7f657c06e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.419+0000 7f6597df4640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:6800/1629025463,v1:172.21.15.46:6801/1629025463] conn(0x7f657c06c300 0x7f657c06e720 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.419+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f657c0664a0 msgr2=0x7f657c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.427 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.419+0000 7f6597df4640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f657c0664a0 0x7f657c068960 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f658800b9d0 tx=0x7f6588008040 comp rx=0 tx=0).stop 2023-12-16T21:43:45.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.419+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900aebd0 msgr2=0x7f659014c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.423+0000 7f6597df4640 1 --2- 172.21.15.46:0/257418947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65900aebd0 0x7f659014c610 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f658400d7d0 tx=0x7f658400dc90 comp rx=0 tx=0).stop 2023-12-16T21:43:45.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.423+0000 7f6596391640 1 -- 172.21.15.46:0/257418947 reap_dead start 2023-12-16T21:43:45.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.423+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 shutdown_connections 2023-12-16T21:43:45.428 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.423+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 >> 172.21.15.46:0/257418947 conn(0x7f65900b4890 msgr2=0x7f6590066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:45.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.423+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 shutdown_connections 2023-12-16T21:43:45.429 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.423+0000 7f6597df4640 1 -- 172.21.15.46:0/257418947 wait complete. 2023-12-16T21:43:45.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-16T21:43:45.558 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d2d2c6640 1 Processor -- start 2023-12-16T21:43:45.559 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d2d2c6640 1 -- start start 2023-12-16T21:43:45.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d2d2c6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 0x7f6d280b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d2d2c6640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d280b7490 con 0x7f6d280b3bc0 2023-12-16T21:43:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d2d2c6640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d28144520 con 0x7f6d28145130 2023-12-16T21:43:45.565 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d2d2c6640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280b4a00 con 0x7f6d280b70b0 2023-12-16T21:43:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d267fc640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f6d280b3bc0 0x7f6d28144a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46480/0 (socket says 172.21.15.46:46480) 2023-12-16T21:43:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d267fc640 1 -- 172.21.15.46:0/2096514269 learned_addr learned my addr 172.21.15.46:0/2096514269 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 0x7f6d280b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456294498 0 0) 0x7f6d280b7490 con 0x7f6d280b3bc0 2023-12-16T21:43:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d10003530 con 0x7f6d280b3bc0 2023-12-16T21:43:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 401615726 0 0) 0x7f6d10003530 con 0x7f6d280b3bc0 2023-12-16T21:43:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 msgr2=0x7f6d280b5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 0x7f6d280b5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 >> v1:172.21.15.161:6789/0 conn(0x7f6d28145130 legacy=0x7f6d2814edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d280b4ce0 con 0x7f6d280b3bc0 2023-12-16T21:43:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f6d1c001680 con 0x7f6d280b3bc0 2023-12-16T21:43:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 0x7f6d280b5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 >> v1:172.21.15.46:6789/0 conn(0x7f6d280b3bc0 legacy=0x7f6d28144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d100039b0 0x7f6d10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 0x7f6d100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d10004d10 0x7f6d10005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.569 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d280b4a00 con 0x7f6d100039b0 2023-12-16T21:43:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1000fdf0 con 0x7f6d10004d10 2023-12-16T21:43:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1000ffb0 con 0x7f6d280b70b0 2023-12-16T21:43:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d100039b0 0x7f6d10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.570 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 -- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 msgr2=0x7f6d100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:45.571 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d280b70b0 0x7f6d100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.572 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 -- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d10004d10 msgr2=0x7f6d10005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d277fe640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d10004d10 0x7f6d10005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d10004d10 0x7f6d10005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.573 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d26ffd640 1 -- 172.21.15.46:0/2096514269 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d18009420 con 0x7f6d100039b0 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.555+0000 7f6d277fe640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d10004d10 0x7f6d10005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d100039b0 0x7f6d10003d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009cb0 tx=0x7f6d1801cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f9388c0dfdc8dc server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d180a5610 con 0x7f6d100039b0 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d180a5770 con 0x7f6d100039b0 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d25ffb640 1 -- 172.21.15.46:0/2096514269 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d180a5b20 con 0x7f6d100039b0 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/2096514269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d100039b0 msgr2=0x7f6d10003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 --2- 172.21.15.46:0/2096514269 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d100039b0 0x7f6d10003d90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009cb0 tx=0x7f6d1801cc30 comp rx=0 tx=0).stop 2023-12-16T21:43:45.574 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d277fe640 1 -- 172.21.15.46:0/2096514269 reap_dead start 2023-12-16T21:43:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/2096514269 shutdown_connections 2023-12-16T21:43:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/2096514269 >> 172.21.15.46:0/2096514269 conn(0x7f6d2806a980 msgr2=0x7f6d2806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/2096514269 shutdown_connections 2023-12-16T21:43:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/2096514269 wait complete. 2023-12-16T21:43:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 Processor -- start 2023-12-16T21:43:45.575 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.559+0000 7f6d2d2c6640 1 -- start start 2023-12-16T21:43:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d100039b0 0x7f6d281dd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d281dde50 0x7f6d28137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d281380e0 0x7f6d2813a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28145680 con 0x7f6d281380e0 2023-12-16T21:43:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281de310 con 0x7f6d281dde50 2023-12-16T21:43:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281de5f0 con 0x7f6d100039b0 2023-12-16T21:43:45.578 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d281380e0 0x7f6d2813a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d281380e0 0x7f6d2813a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35598/0 (socket says 172.21.15.46:35598) 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 -- 172.21.15.46:0/1812447466 learned_addr learned my addr 172.21.15.46:0/1812447466 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d100039b0 0x7f6d281dd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d267fc640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d281dde50 0x7f6d28137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 -- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d100039b0 msgr2=0x7f6d281dd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d100039b0 0x7f6d281dd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 -- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d281dde50 msgr2=0x7f6d28137ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.579 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d281dde50 0x7f6d28137ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 -- 172.21.15.46:0/1812447466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d280b0800 con 0x7f6d281380e0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d277fe640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d281380e0 0x7f6d2813a1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6d28148b30 tx=0x7f6d2000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d0f7fe640 1 -- 172.21.15.46:0/1812447466 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d20003d90 con 0x7f6d281380e0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d0f7fe640 1 -- 172.21.15.46:0/1812447466 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2000de50 con 0x7f6d281380e0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d0f7fe640 1 -- 172.21.15.46:0/1812447466 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d200052b0 con 0x7f6d281380e0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d2813d9c0 con 0x7f6d281380e0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2813a730 con 0x7f6d281380e0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d28148960 con 0x7f6d281380e0 2023-12-16T21:43:45.580 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d0f7fe640 1 -- 172.21.15.46:0/1812447466 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f6d20005410 con 0x7f6d281380e0 2023-12-16T21:43:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d0f7fe640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6d14066470 0x7f6d14068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d0f7fe640 1 -- 172.21.15.46:0/1812447466 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d200834c0 con 0x7f6d281380e0 2023-12-16T21:43:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.563+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6d14066470 0x7f6d14068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.567+0000 7f6d26ffd640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6d14066470 0x7f6d14068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6d180af000 tx=0x7f6d180046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.581 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.567+0000 7f6d0f7fe640 1 -- 172.21.15.46:0/1812447466 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f6d20050d80 con 0x7f6d281380e0 2023-12-16T21:43:45.728 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.723+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d280689d0 con 0x7f6d14066470 2023-12-16T21:43:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d0f7fe640 1 -- 172.21.15.46:0/1812447466 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6d280689d0 con 0x7f6d14066470 2023-12-16T21:43:45.729 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 0 to deep-scrub 2023-12-16T21:43:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6d14066470 msgr2=0x7f6d14068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6d14066470 0x7f6d14068930 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6d180af000 tx=0x7f6d180046d0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d281380e0 msgr2=0x7f6d2813a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d281380e0 0x7f6d2813a1f0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6d28148b30 tx=0x7f6d2000dc00 comp rx=0 tx=0).stop 2023-12-16T21:43:45.731 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 shutdown_connections 2023-12-16T21:43:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f6d14066470 0x7f6d14068930 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6d281380e0 0x7f6d2813a1f0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f6d281dde50 0x7f6d28137ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 --2- 172.21.15.46:0/1812447466 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f6d100039b0 0x7f6d281dd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 >> 172.21.15.46:0/1812447466 conn(0x7f6d2806a980 msgr2=0x7f6d280ba5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 shutdown_connections 2023-12-16T21:43:45.732 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.727+0000 7f6d2d2c6640 1 -- 172.21.15.46:0/1812447466 wait complete. 2023-12-16T21:43:45.748 INFO:tasks.ceph:Scrubbing osd.1 2023-12-16T21:43:45.748 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:45.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad92c5640 1 Processor -- start 2023-12-16T21:43:45.867 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad92c5640 1 -- start start 2023-12-16T21:43:45.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad92c5640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fdad4071ad0 0x7fdad40b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad92c5640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdad40b3d70 0x7fdad414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.868 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad92c5640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40bad10 con 0x7fdad40722a0 2023-12-16T21:43:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad92c5640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad4071020 con 0x7fdad40b3d70 2023-12-16T21:43:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad92c5640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad40b20f0 con 0x7fdad4071ad0 2023-12-16T21:43:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad37fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fdad40b3d70 0x7fdad414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad2ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7fdad40722a0 0x7fdad4144520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46488/0 (socket says 172.21.15.46:46488) 2023-12-16T21:43:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad2ffd640 1 -- 172.21.15.46:0/1140748795 learned_addr learned my addr 172.21.15.46:0/1140748795 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdaca7fc640 1 --2- 172.21.15.46:0/1140748795 >> v2:172.21.15.161:3300/0 conn(0x7fdad4071ad0 0x7fdad40b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.869 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad37fe640 1 -- 172.21.15.46:0/1140748795 >> v2:172.21.15.161:3300/0 conn(0x7fdad4071ad0 msgr2=0x7fdad40b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:45.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad37fe640 1 --2- 172.21.15.46:0/1140748795 >> v2:172.21.15.161:3300/0 conn(0x7fdad4071ad0 0x7fdad40b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad37fe640 1 -- 172.21.15.46:0/1140748795 >> v1:172.21.15.46:6789/0 conn(0x7fdad40722a0 legacy=0x7fdad4144520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad37fe640 1 -- 172.21.15.46:0/1140748795 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad4070a60 con 0x7fdad40b3d70 2023-12-16T21:43:45.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.863+0000 7fdad27fc640 1 -- 172.21.15.46:0/1140748795 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719342300 0 0) 0x7fdad40bad10 con 0x7fdad40722a0 2023-12-16T21:43:45.871 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad37fe640 1 --2- 172.21.15.46:0/1140748795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad40b3d70 0x7fdad414ee00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b1310 tx=0x7fdacc01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9137e3211363eb0a server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad27fc640 1 -- 172.21.15.46:0/1140748795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0a5740 con 0x7fdad40b3d70 2023-12-16T21:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad27fc640 1 -- 172.21.15.46:0/1140748795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc0a58a0 con 0x7fdad40b3d70 2023-12-16T21:43:45.872 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad27fc640 1 -- 172.21.15.46:0/1140748795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc0a5c30 con 0x7fdad40b3d70 2023-12-16T21:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 -- 172.21.15.46:0/1140748795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad40b3d70 msgr2=0x7fdad414ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 --2- 172.21.15.46:0/1140748795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad40b3d70 0x7fdad414ee00 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdad40b1310 tx=0x7fdacc01c880 comp rx=0 tx=0).stop 2023-12-16T21:43:45.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 -- 172.21.15.46:0/1140748795 shutdown_connections 2023-12-16T21:43:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 --2- 172.21.15.46:0/1140748795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad40b3d70 0x7fdad414ee00 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 --2- 172.21.15.46:0/1140748795 >> v2:172.21.15.161:3300/0 conn(0x7fdad4071ad0 0x7fdad40b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 -- 172.21.15.46:0/1140748795 >> 172.21.15.46:0/1140748795 conn(0x7fdad406ab80 msgr2=0x7fdad40b9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:45.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 -- 172.21.15.46:0/1140748795 shutdown_connections 2023-12-16T21:43:45.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 -- 172.21.15.46:0/1140748795 wait complete. 2023-12-16T21:43:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 Processor -- start 2023-12-16T21:43:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.867+0000 7fdad92c5640 1 -- start start 2023-12-16T21:43:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4071ad0 0x7fdad40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.878 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad40722a0 0x7fdad4136220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdad40b3d70 0x7fdad4136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad41377d0 con 0x7fdad4071ad0 2023-12-16T21:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad413e090 con 0x7fdad40722a0 2023-12-16T21:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406d5a0 con 0x7fdad40b3d70 2023-12-16T21:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4071ad0 0x7fdad40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4071ad0 0x7fdad40acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35626/0 (socket says 172.21.15.46:35626) 2023-12-16T21:43:45.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 -- 172.21.15.46:0/597774122 learned_addr learned my addr 172.21.15.46:0/597774122 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad37fe640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdad40b3d70 0x7fdad4136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdaca7fc640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad40722a0 0x7fdad4136220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 -- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdad40b3d70 msgr2=0x7fdad4136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fdad40b3d70 0x7fdad4136760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 -- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad40722a0 msgr2=0x7fdad4136220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fdad40722a0 0x7fdad4136220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 -- 172.21.15.46:0/597774122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc009a70 con 0x7fdad4071ad0 2023-12-16T21:43:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad2ffd640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4071ad0 0x7fdad40acb50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c520 tx=0x7fdac400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac40074e0 con 0x7fdad4071ad0 2023-12-16T21:43:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac4007640 con 0x7fdad4071ad0 2023-12-16T21:43:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac40049f0 con 0x7fdad4071ad0 2023-12-16T21:43:45.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad41420e0 con 0x7fdad4071ad0 2023-12-16T21:43:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad413dd20 con 0x7fdad4071ad0 2023-12-16T21:43:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdad4143fa0 con 0x7fdad4071ad0 2023-12-16T21:43:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fdac401b360 con 0x7fdad4071ad0 2023-12-16T21:43:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdab4066470 0x7fdab4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdac40833f0 con 0x7fdad4071ad0 2023-12-16T21:43:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7fdab406c2d0 0x7fdab406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 --> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdab406eda0 con 0x7fdab406c2d0 2023-12-16T21:43:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdac4083840 con 0x7fdad4071ad0 2023-12-16T21:43:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdaca7fc640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdab4066470 0x7fdab4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad37fe640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7fdab406c2d0 0x7fdab406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:45.886 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.871+0000 7fdad37fe640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7fdab406c2d0 0x7fdab406e6f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.887 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.875+0000 7fdaca7fc640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdab4066470 0x7fdab4068930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdad413edc0 tx=0x7fdabc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:45.888 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.875+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== osd.1 v2:172.21.15.46:6824/491328631 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fdab406eda0 con 0x7fdab406c2d0 2023-12-16T21:43:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 --> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdad4144c30 con 0x7fdab406c2d0 2023-12-16T21:43:45.890 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdacbfff640 1 -- 172.21.15.46:0/597774122 <== osd.1 v2:172.21.15.46:6824/491328631 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fdacc0a5540 con 0x7fdab406c2d0 2023-12-16T21:43:45.891 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:45.891 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:45.891 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7fdab406c2d0 msgr2=0x7fdab406e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:6824/491328631,v1:172.21.15.46:6825/491328631] conn(0x7fdab406c2d0 0x7fdab406e6f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdab4066470 msgr2=0x7fdab4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fdab4066470 0x7fdab4068930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fdad413edc0 tx=0x7fdabc008040 comp rx=0 tx=0).stop 2023-12-16T21:43:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4071ad0 msgr2=0x7fdad40acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 --2- 172.21.15.46:0/597774122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdad4071ad0 0x7fdad40acb50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdac400c520 tx=0x7fdac400c9e0 comp rx=0 tx=0).stop 2023-12-16T21:43:45.893 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad37fe640 1 -- 172.21.15.46:0/597774122 reap_dead start 2023-12-16T21:43:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 shutdown_connections 2023-12-16T21:43:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 >> 172.21.15.46:0/597774122 conn(0x7fdad406ab80 msgr2=0x7fdad40b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 shutdown_connections 2023-12-16T21:43:45.894 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:45.887+0000 7fdad92c5640 1 -- 172.21.15.46:0/597774122 wait complete. 2023-12-16T21:43:45.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-16T21:43:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f259e8d8640 1 Processor -- start 2023-12-16T21:43:46.014 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f259e8d8640 1 -- start start 2023-12-16T21:43:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f259e8d8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2598073710 0x7f2598147b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.015 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f259e8d8640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25980af2f0 0x7f25980b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f259e8d8640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25980ad630 con 0x7f25980b4430 2023-12-16T21:43:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f259e8d8640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980ae750 con 0x7f25980af2f0 2023-12-16T21:43:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f259e8d8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2598073af0 con 0x7f2598073710 2023-12-16T21:43:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f25977fe640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f25980af2f0 0x7f25980b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f2597fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2598073710 0x7f2598147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f2597fff640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2598073710 0x7f2598147b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:45492/0 (socket says 172.21.15.46:45492) 2023-12-16T21:43:46.016 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f2597fff640 1 -- 172.21.15.46:0/226768984 learned_addr learned my addr 172.21.15.46:0/226768984 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f2597fff640 1 -- 172.21.15.46:0/226768984 >> v2:172.21.15.161:3300/0 conn(0x7f25980af2f0 msgr2=0x7f25980b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f2597fff640 1 --2- 172.21.15.46:0/226768984 >> v2:172.21.15.161:3300/0 conn(0x7f25980af2f0 0x7f25980b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f2597fff640 1 -- 172.21.15.46:0/226768984 >> v1:172.21.15.161:6789/0 conn(0x7f25980b4430 legacy=0x7f25981412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.017 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f2597fff640 1 -- 172.21.15.46:0/226768984 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f259806a7d0 con 0x7f2598073710 2023-12-16T21:43:46.018 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.011+0000 7f25977fe640 1 --2- 172.21.15.46:0/226768984 >> v2:172.21.15.161:3300/0 conn(0x7f25980af2f0 0x7f25980b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f2597fff640 1 --2- 172.21.15.46:0/226768984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2598073710 0x7f2598147b80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f258c009870 tx=0x7f258c01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17d10f62dd4e75b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f2596ffd640 1 -- 172.21.15.46:0/226768984 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0ab680 con 0x7f2598073710 2023-12-16T21:43:46.019 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f2596ffd640 1 -- 172.21.15.46:0/226768984 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c0ab7e0 con 0x7f2598073710 2023-12-16T21:43:46.020 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f2596ffd640 1 -- 172.21.15.46:0/226768984 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0abb70 con 0x7f2598073710 2023-12-16T21:43:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 -- 172.21.15.46:0/226768984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2598073710 msgr2=0x7f2598147b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 --2- 172.21.15.46:0/226768984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2598073710 0x7f2598147b80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f258c009870 tx=0x7f258c01c480 comp rx=0 tx=0).stop 2023-12-16T21:43:46.022 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 -- 172.21.15.46:0/226768984 shutdown_connections 2023-12-16T21:43:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 --2- 172.21.15.46:0/226768984 >> v2:172.21.15.161:3300/0 conn(0x7f25980af2f0 0x7f25980b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 --2- 172.21.15.46:0/226768984 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f2598073710 0x7f2598147b80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 -- 172.21.15.46:0/226768984 >> 172.21.15.46:0/226768984 conn(0x7f2598099a70 msgr2=0x7f2598099e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 -- 172.21.15.46:0/226768984 shutdown_connections 2023-12-16T21:43:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.015+0000 7f259e8d8640 1 -- 172.21.15.46:0/226768984 wait complete. 2023-12-16T21:43:46.023 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 Processor -- start 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 -- start start 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2598073710 0x7f259813bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25980af2f0 0x7f259813ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25980b4430 0x7f259813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25981402f0 con 0x7f2598073710 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598133810 con 0x7f25980af2f0 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25981e1190 con 0x7f25980b4430 2023-12-16T21:43:46.024 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2598073710 0x7f259813bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2598073710 0x7f259813bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35634/0 (socket says 172.21.15.46:35634) 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 -- 172.21.15.46:0/4165776792 learned_addr learned my addr 172.21.15.46:0/4165776792 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f25977fe640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25980af2f0 0x7f259813ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259ce75640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25980b4430 0x7f259813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 -- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25980b4430 msgr2=0x7f259813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25980b4430 0x7f259813f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 -- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25980af2f0 msgr2=0x7f259813ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25980af2f0 0x7f259813ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.025 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 -- 172.21.15.46:0/4165776792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2588009420 con 0x7f2598073710 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f25977fe640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25980af2f0 0x7f259813ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259ce75640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25980b4430 0x7f259813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2597fff640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2598073710 0x7f259813bb70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad040 tx=0x7f258c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2594ff9640 1 -- 172.21.15.46:0/4165776792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0ab680 con 0x7f2598073710 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2594ff9640 1 -- 172.21.15.46:0/4165776792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f258c0b4800 con 0x7f2598073710 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c0094d0 con 0x7f2598073710 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2594ff9640 1 -- 172.21.15.46:0/4165776792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f258c0b4bb0 con 0x7f2598073710 2023-12-16T21:43:46.026 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25981e0c70 con 0x7f2598073710 2023-12-16T21:43:46.027 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25981e0e60 con 0x7f2598073710 2023-12-16T21:43:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2594ff9640 1 -- 172.21.15.46:0/4165776792 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f258c0b4d10 con 0x7f2598073710 2023-12-16T21:43:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2594ff9640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2584066470 0x7f2584068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f2594ff9640 1 -- 172.21.15.46:0/4165776792 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f258c11ea20 con 0x7f2598073710 2023-12-16T21:43:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.019+0000 7f25977fe640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2584066470 0x7f2584068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.023+0000 7f25977fe640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2584066470 0x7f2584068930 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f259813e5c0 tx=0x7f2588008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.028 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.023+0000 7f2594ff9640 1 -- 172.21.15.46:0/4165776792 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f258c0ec1b0 con 0x7f2598073710 2023-12-16T21:43:46.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.171+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f259813dec0 con 0x7f2584066470 2023-12-16T21:43:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f2594ff9640 1 -- 172.21.15.46:0/4165776792 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f259813dec0 con 0x7f2584066470 2023-12-16T21:43:46.177 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 1 to deep-scrub 2023-12-16T21:43:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2584066470 msgr2=0x7f2584068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2584066470 0x7f2584068930 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f259813e5c0 tx=0x7f2588008040 comp rx=0 tx=0).stop 2023-12-16T21:43:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2598073710 msgr2=0x7f259813bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2598073710 0x7f259813bb70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f258c0ad040 tx=0x7f258c01acc0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 shutdown_connections 2023-12-16T21:43:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2584066470 0x7f2584068930 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f25980b4430 0x7f259813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f25980af2f0 0x7f259813ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 --2- 172.21.15.46:0/4165776792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2598073710 0x7f259813bb70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 >> 172.21.15.46:0/4165776792 conn(0x7f2598099a70 msgr2=0x7f259813d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 shutdown_connections 2023-12-16T21:43:46.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.175+0000 7f259e8d8640 1 -- 172.21.15.46:0/4165776792 wait complete. 2023-12-16T21:43:46.194 INFO:tasks.ceph:Scrubbing osd.2 2023-12-16T21:43:46.195 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd3541e9640 1 Processor -- start 2023-12-16T21:43:46.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd3541e9640 1 -- start start 2023-12-16T21:43:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd3541e9640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7fd34c06c920 0x7fd34c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.324 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd3541e9640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd34c0afa00 0x7fd34c144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd3541e9640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34c1453d0 con 0x7fd34c144fa0 2023-12-16T21:43:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd3541e9640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c0b3cd0 con 0x7fd34c0afa00 2023-12-16T21:43:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd3541e9640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34c0b1ea0 con 0x7fd34c06c920 2023-12-16T21:43:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351784640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd34c0afa00 0x7fd34c144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351784640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7fd34c0afa00 0x7fd34c144a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35644/0 (socket says 172.21.15.46:35644) 2023-12-16T21:43:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351784640 1 -- 172.21.15.46:0/2209824377 learned_addr learned my addr 172.21.15.46:0/2209824377 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351f85640 1 --2- 172.21.15.46:0/2209824377 >> v2:172.21.15.161:3300/0 conn(0x7fd34c06c920 0x7fd34c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351784640 1 -- 172.21.15.46:0/2209824377 >> v2:172.21.15.161:3300/0 conn(0x7fd34c06c920 msgr2=0x7fd34c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351784640 1 --2- 172.21.15.46:0/2209824377 >> v2:172.21.15.161:3300/0 conn(0x7fd34c06c920 0x7fd34c0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351784640 1 -- 172.21.15.46:0/2209824377 >> v1:172.21.15.161:6789/0 conn(0x7fd34c144fa0 legacy=0x7fd34c0b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351784640 1 -- 172.21.15.46:0/2209824377 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd34c0b1b70 con 0x7fd34c0afa00 2023-12-16T21:43:46.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.319+0000 7fd351f85640 1 --2- 172.21.15.46:0/2209824377 >> v2:172.21.15.161:3300/0 conn(0x7fd34c06c920 0x7fd34c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd351784640 1 --2- 172.21.15.46:0/2209824377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0afa00 0x7fd34c144a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0af0a0 tx=0x7fd34001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ea0c572df9b892 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.327 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd350f83640 1 -- 172.21.15.46:0/2209824377 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3400ac640 con 0x7fd34c0afa00 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd350f83640 1 -- 172.21.15.46:0/2209824377 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3400ac7a0 con 0x7fd34c0afa00 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd350f83640 1 -- 172.21.15.46:0/2209824377 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3400acb30 con 0x7fd34c0afa00 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 -- 172.21.15.46:0/2209824377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0afa00 msgr2=0x7fd34c144a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 --2- 172.21.15.46:0/2209824377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0afa00 0x7fd34c144a60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0af0a0 tx=0x7fd34001c4c0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 -- 172.21.15.46:0/2209824377 shutdown_connections 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 --2- 172.21.15.46:0/2209824377 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c0afa00 0x7fd34c144a60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 --2- 172.21.15.46:0/2209824377 >> v2:172.21.15.161:3300/0 conn(0x7fd34c06c920 0x7fd34c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 -- 172.21.15.46:0/2209824377 >> 172.21.15.46:0/2209824377 conn(0x7fd34c0b4890 msgr2=0x7fd34c0b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 -- 172.21.15.46:0/2209824377 shutdown_connections 2023-12-16T21:43:46.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 -- 172.21.15.46:0/2209824377 wait complete. 2023-12-16T21:43:46.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 Processor -- start 2023-12-16T21:43:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.323+0000 7fd3541e9640 1 -- start start 2023-12-16T21:43:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd34c06c920 0x7fd34c09a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd34c0afa00 0x7fd34c09a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c144fa0 0x7fd34c1dce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c142020 con 0x7fd34c144fa0 2023-12-16T21:43:46.333 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c09bb20 con 0x7fd34c0afa00 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34c1419d0 con 0x7fd34c06c920 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c144fa0 0x7fd34c1dce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c144fa0 0x7fd34c1dce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35650/0 (socket says 172.21.15.46:35650) 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 -- 172.21.15.46:0/1860392856 learned_addr learned my addr 172.21.15.46:0/1860392856 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd351784640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd34c0afa00 0x7fd34c09a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd351f85640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd34c06c920 0x7fd34c09a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 -- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd34c06c920 msgr2=0x7fd34c09a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.334 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd34c06c920 0x7fd34c09a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 -- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd34c0afa00 msgr2=0x7fd34c09a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd34c0afa00 0x7fd34c09a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 -- 172.21.15.46:0/1860392856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400094d0 con 0x7fd34c144fa0 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd351784640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7fd34c0afa00 0x7fd34c09a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd351f85640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7fd34c06c920 0x7fd34c09a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd352786640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c144fa0 0x7fd34c1dce10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b990 tx=0x7fd34400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd34400cba0 con 0x7fd34c144fa0 2023-12-16T21:43:46.335 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd34400cd00 con 0x7fd34c144fa0 2023-12-16T21:43:46.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd344014680 con 0x7fd34c144fa0 2023-12-16T21:43:46.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34c141cb0 con 0x7fd34c144fa0 2023-12-16T21:43:46.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34c1407f0 con 0x7fd34c144fa0 2023-12-16T21:43:46.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd34c144400 con 0x7fd34c144fa0 2023-12-16T21:43:46.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7fd34401d3b0 con 0x7fd34c144fa0 2023-12-16T21:43:46.336 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd330066470 0x7fd330068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd344084160 con 0x7fd34c144fa0 2023-12-16T21:43:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7fd33006c2d0 0x7fd33006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 --> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd33006eda0 con 0x7fd33006c2d0 2023-12-16T21:43:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3440845b0 con 0x7fd34c144fa0 2023-12-16T21:43:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd351f85640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd330066470 0x7fd330068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.327+0000 7fd351784640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7fd33006c2d0 0x7fd33006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.337 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.331+0000 7fd351784640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7fd33006c2d0 0x7fd33006e6f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.331+0000 7fd351f85640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd330066470 0x7fd330068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd34c13a3f0 tx=0x7fd348008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.338 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.331+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== osd.2 v2:172.21.15.46:6808/3526272141 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd33006eda0 con 0x7fd33006c2d0 2023-12-16T21:43:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 --> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd34c0689d0 con 0x7fd33006c2d0 2023-12-16T21:43:46.347 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd33e7fc640 1 -- 172.21.15.46:0/1860392856 <== osd.2 v2:172.21.15.46:6808/3526272141 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd34c0689d0 con 0x7fd33006c2d0 2023-12-16T21:43:46.348 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:46.348 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:46.348 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7fd33006c2d0 msgr2=0x7fd33006e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:6808/3526272141,v1:172.21.15.46:6810/3526272141] conn(0x7fd33006c2d0 0x7fd33006e6f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd330066470 msgr2=0x7fd330068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7fd330066470 0x7fd330068930 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd34c13a3f0 tx=0x7fd348008040 comp rx=0 tx=0).stop 2023-12-16T21:43:46.349 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c144fa0 msgr2=0x7fd34c1dce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 --2- 172.21.15.46:0/1860392856 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd34c144fa0 0x7fd34c1dce10 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd34400b990 tx=0x7fd34400be50 comp rx=0 tx=0).stop 2023-12-16T21:43:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd352786640 1 -- 172.21.15.46:0/1860392856 reap_dead start 2023-12-16T21:43:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 shutdown_connections 2023-12-16T21:43:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 >> 172.21.15.46:0/1860392856 conn(0x7fd34c0b4890 msgr2=0x7fd34c0b5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 shutdown_connections 2023-12-16T21:43:46.350 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.343+0000 7fd3541e9640 1 -- 172.21.15.46:0/1860392856 wait complete. 2023-12-16T21:43:46.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-16T21:43:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4bae050640 1 Processor -- start 2023-12-16T21:43:46.473 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4bae050640 1 -- start start 2023-12-16T21:43:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4bae050640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4ba80af4f0 0x7f4ba809caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4bae050640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ba8148740 0x7f4ba809cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4bae050640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba80b1800 con 0x7f4ba80722c0 2023-12-16T21:43:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4bae050640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80b1610 con 0x7f4ba8148740 2023-12-16T21:43:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4bae050640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ba80b1aa0 con 0x7f4ba80af4f0 2023-12-16T21:43:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba7fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ba8148740 0x7f4ba809cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba6ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4ba80722c0 0x7f4ba8148030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46498/0 (socket says 172.21.15.46:46498) 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba6ffd640 1 -- 172.21.15.46:0/1136947603 learned_addr learned my addr 172.21.15.46:0/1136947603 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba7fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ba8148740 0x7f4ba809cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35656/0 (socket says 172.21.15.46:35656) 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba77fe640 1 --2- 172.21.15.46:0/1136947603 >> v2:172.21.15.161:3300/0 conn(0x7f4ba80af4f0 0x7f4ba809caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba7fff640 1 -- 172.21.15.46:0/1136947603 >> v2:172.21.15.161:3300/0 conn(0x7f4ba80af4f0 msgr2=0x7f4ba809caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba7fff640 1 --2- 172.21.15.46:0/1136947603 >> v2:172.21.15.161:3300/0 conn(0x7f4ba80af4f0 0x7f4ba809caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba7fff640 1 -- 172.21.15.46:0/1136947603 >> v1:172.21.15.46:6789/0 conn(0x7f4ba80722c0 legacy=0x7f4ba8148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba7fff640 1 -- 172.21.15.46:0/1136947603 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba8147b10 con 0x7f4ba8148740 2023-12-16T21:43:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba77fe640 1 --2- 172.21.15.46:0/1136947603 >> v2:172.21.15.161:3300/0 conn(0x7f4ba80af4f0 0x7f4ba809caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba7fff640 1 --2- 172.21.15.46:0/1136947603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba8148740 0x7f4ba809cfe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80b5c60 tx=0x7f4b9801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518f55f3ec6c0eca server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba67fc640 1 -- 172.21.15.46:0/1136947603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980a5770 con 0x7f4ba8148740 2023-12-16T21:43:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.471+0000 7f4ba67fc640 1 -- 172.21.15.46:0/1136947603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b980a58d0 con 0x7f4ba8148740 2023-12-16T21:43:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba67fc640 1 -- 172.21.15.46:0/1136947603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b980a5c60 con 0x7f4ba8148740 2023-12-16T21:43:46.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- 172.21.15.46:0/1136947603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba8148740 msgr2=0x7f4ba809cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 --2- 172.21.15.46:0/1136947603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba8148740 0x7f4ba809cfe0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4ba80b5c60 tx=0x7f4b9801c880 comp rx=0 tx=0).stop 2023-12-16T21:43:46.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- 172.21.15.46:0/1136947603 shutdown_connections 2023-12-16T21:43:46.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 --2- 172.21.15.46:0/1136947603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba8148740 0x7f4ba809cfe0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 --2- 172.21.15.46:0/1136947603 >> v2:172.21.15.161:3300/0 conn(0x7f4ba80af4f0 0x7f4ba809caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- 172.21.15.46:0/1136947603 >> 172.21.15.46:0/1136947603 conn(0x7f4ba806a580 msgr2=0x7f4ba806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- 172.21.15.46:0/1136947603 shutdown_connections 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- 172.21.15.46:0/1136947603 wait complete. 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 Processor -- start 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- start start 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba80722c0 0x7f4ba80ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba80af4f0 0x7f4ba80ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba8148740 0x7f4ba81ded20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80aee60 con 0x7f4ba80722c0 2023-12-16T21:43:46.482 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba81432a0 con 0x7f4ba8148740 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80adee0 con 0x7f4ba80af4f0 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba80722c0 0x7f4ba80ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba80722c0 0x7f4ba80ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35662/0 (socket says 172.21.15.46:35662) 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 -- 172.21.15.46:0/2947367797 learned_addr learned my addr 172.21.15.46:0/2947367797 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba7fff640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba8148740 0x7f4ba81ded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 -- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba80af4f0 msgr2=0x7f4ba80ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba6ffd640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba80af4f0 0x7f4ba80ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba80af4f0 0x7f4ba80ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 -- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba8148740 msgr2=0x7f4ba81ded20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba8148740 0x7f4ba81ded20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 -- 172.21.15.46:0/2947367797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b98009a70 con 0x7f4ba80722c0 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba7fff640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba8148740 0x7f4ba81ded20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4ba77fe640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba80722c0 0x7f4ba80ae7d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c7f0 tx=0x7f4ba000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4b8ffff640 1 -- 172.21.15.46:0/2947367797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba000cea0 con 0x7f4ba80722c0 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4b8ffff640 1 -- 172.21.15.46:0/2947367797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba0007a30 con 0x7f4ba80722c0 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba80ae1c0 con 0x7f4ba80722c0 2023-12-16T21:43:46.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4b8ffff640 1 -- 172.21.15.46:0/2947367797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0007de0 con 0x7f4ba80722c0 2023-12-16T21:43:46.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.475+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba80acdb0 con 0x7f4ba80722c0 2023-12-16T21:43:46.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.479+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba80679e0 con 0x7f4ba80722c0 2023-12-16T21:43:46.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.479+0000 7f4b8ffff640 1 -- 172.21.15.46:0/2947367797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f4ba00113f0 con 0x7f4ba80722c0 2023-12-16T21:43:46.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.479+0000 7f4b8ffff640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4b940664a0 0x7f4b94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.479+0000 7f4b8ffff640 1 -- 172.21.15.46:0/2947367797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4ba0083c20 con 0x7f4ba80722c0 2023-12-16T21:43:46.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.479+0000 7f4ba6ffd640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4b940664a0 0x7f4b94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.486 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.479+0000 7f4ba6ffd640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4b940664a0 0x7f4b94068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ba81e20a0 tx=0x7f4b9c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.487 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.483+0000 7f4b8ffff640 1 -- 172.21.15.46:0/2947367797 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f4ba0051430 con 0x7f4ba80722c0 2023-12-16T21:43:46.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.631+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ba80ae400 con 0x7f4b940664a0 2023-12-16T21:43:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.631+0000 7f4b8ffff640 1 -- 172.21.15.46:0/2947367797 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4ba80ae400 con 0x7f4b940664a0 2023-12-16T21:43:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 2 to deep-scrub 2023-12-16T21:43:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4b940664a0 msgr2=0x7f4b94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4b940664a0 0x7f4b94068960 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4ba81e20a0 tx=0x7f4b9c009040 comp rx=0 tx=0).stop 2023-12-16T21:43:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba80722c0 msgr2=0x7f4ba80ae7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba80722c0 0x7f4ba80ae7d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c7f0 tx=0x7f4ba000ccb0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 shutdown_connections 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4b940664a0 0x7f4b94068960 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ba8148740 0x7f4ba81ded20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ba80af4f0 0x7f4ba80ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 --2- 172.21.15.46:0/2947367797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ba80722c0 0x7f4ba80ae7d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 >> 172.21.15.46:0/2947367797 conn(0x7f4ba806a580 msgr2=0x7f4ba80b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 shutdown_connections 2023-12-16T21:43:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.635+0000 7f4bae050640 1 -- 172.21.15.46:0/2947367797 wait complete. 2023-12-16T21:43:46.653 INFO:tasks.ceph:Scrubbing osd.3 2023-12-16T21:43:46.654 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cd5774640 1 Processor -- start 2023-12-16T21:43:46.770 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cd5774640 1 -- start start 2023-12-16T21:43:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cd5774640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4cd00ae0f0 0x7f4cd0146be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cd5774640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4cd0147120 0x7f4cd01475d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cd5774640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd0072550 con 0x7f4cd00b29f0 2023-12-16T21:43:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cd5774640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00af610 con 0x7f4cd0147120 2023-12-16T21:43:46.772 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cd5774640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd00ae750 con 0x7f4cd00ae0f0 2023-12-16T21:43:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cceffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f4cd00b29f0 0x7f4cd00b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46502/0 (socket says 172.21.15.46:46502) 2023-12-16T21:43:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cceffd640 1 -- 172.21.15.46:0/2175345680 learned_addr learned my addr 172.21.15.46:0/2175345680 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccf7fe640 1 --2- 172.21.15.46:0/2175345680 >> v2:172.21.15.46:3300/0 conn(0x7f4cd0147120 0x7f4cd01475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4cce7fc640 1 --2- 172.21.15.46:0/2175345680 >> v2:172.21.15.161:3300/0 conn(0x7f4cd00ae0f0 0x7f4cd0146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.773 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccdffb640 1 -- 172.21.15.46:0/2175345680 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602500069 0 0) 0x7f4cc0001120 con 0x7f4cd00b29f0 2023-12-16T21:43:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccdffb640 1 -- 172.21.15.46:0/2175345680 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb8003530 con 0x7f4cd00b29f0 2023-12-16T21:43:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccdffb640 1 -- 172.21.15.46:0/2175345680 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1647157734 0 0) 0x7f4cb8003530 con 0x7f4cd00b29f0 2023-12-16T21:43:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccf7fe640 1 -- 172.21.15.46:0/2175345680 >> v2:172.21.15.161:3300/0 conn(0x7f4cd00ae0f0 msgr2=0x7f4cd0146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccf7fe640 1 --2- 172.21.15.46:0/2175345680 >> v2:172.21.15.161:3300/0 conn(0x7f4cd00ae0f0 0x7f4cd0146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccf7fe640 1 -- 172.21.15.46:0/2175345680 >> v1:172.21.15.46:6789/0 conn(0x7f4cd00b29f0 legacy=0x7f4cd00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.774 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.767+0000 7f4ccf7fe640 1 -- 172.21.15.46:0/2175345680 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd00b1180 con 0x7f4cd0147120 2023-12-16T21:43:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4ccf7fe640 1 --2- 172.21.15.46:0/2175345680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01475d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4cd006e1f0 tx=0x7f4cc801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2946f3813d4928e server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4ccdffb640 1 -- 172.21.15.46:0/2175345680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0001810 con 0x7f4cd0147120 2023-12-16T21:43:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4ccdffb640 1 -- 172.21.15.46:0/2175345680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc80a5550 con 0x7f4cd0147120 2023-12-16T21:43:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4ccdffb640 1 -- 172.21.15.46:0/2175345680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc80a58e0 con 0x7f4cd0147120 2023-12-16T21:43:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- 172.21.15.46:0/2175345680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 msgr2=0x7f4cd01475d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 --2- 172.21.15.46:0/2175345680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01475d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4cd006e1f0 tx=0x7f4cc801c710 comp rx=0 tx=0).stop 2023-12-16T21:43:46.780 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- 172.21.15.46:0/2175345680 shutdown_connections 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 --2- 172.21.15.46:0/2175345680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01475d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 --2- 172.21.15.46:0/2175345680 >> v2:172.21.15.161:3300/0 conn(0x7f4cd00ae0f0 0x7f4cd0146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- 172.21.15.46:0/2175345680 >> 172.21.15.46:0/2175345680 conn(0x7f4cd00b4890 msgr2=0x7f4cd00b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- 172.21.15.46:0/2175345680 shutdown_connections 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- 172.21.15.46:0/2175345680 wait complete. 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 Processor -- start 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- start start 2023-12-16T21:43:46.781 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4cd00b29f0 0x7f4cd014da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01deea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0146cf0 0x7f4cd01df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd0070670 con 0x7f4cd0147120 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01460d0 con 0x7f4cd0146cf0 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.771+0000 7f4cd5774640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd01463b0 con 0x7f4cd00b29f0 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01deea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35676/0 (socket says 172.21.15.46:35676) 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 -- 172.21.15.46:0/2432532883 learned_addr learned my addr 172.21.15.46:0/2432532883 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.782 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cceffd640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4cd00b29f0 0x7f4cd014da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 -- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4cd00b29f0 msgr2=0x7f4cd014da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4cd00b29f0 0x7f4cd014da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 -- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0146cf0 msgr2=0x7f4cd01df3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4cd0146cf0 0x7f4cd01df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 -- 172.21.15.46:0/2432532883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8009a70 con 0x7f4cd0147120 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cce7fc640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01deea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e8d0 tx=0x7f4cc400ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4009630 con 0x7f4cd0147120 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc400c940 con 0x7f4cd0147120 2023-12-16T21:43:46.783 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd00af000 con 0x7f4cd0147120 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400ccb0 con 0x7f4cd0147120 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd013adc0 con 0x7f4cd0147120 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4cd00aee80 con 0x7f4cd0147120 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f4cc4013470 con 0x7f4cd0147120 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4cbc0664a0 0x7f4cbc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4cc4083c80 con 0x7f4cd0147120 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7f4cbc06c300 0x7f4cbc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.784 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 --> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4cbc06edd0 con 0x7f4cbc06c300 2023-12-16T21:43:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cc4086340 con 0x7f4cd0147120 2023-12-16T21:43:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4ccf7fe640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7f4cbc06c300 0x7f4cbc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cceffd640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4cbc0664a0 0x7f4cbc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4ccf7fe640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7f4cbc06c300 0x7f4cbc06e720 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.785 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.775+0000 7f4cceffd640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4cbc0664a0 0x7f4cbc068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0001380 tx=0x7f4cc0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.786 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.779+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== osd.3 v2:172.21.15.46:6809/2268558465 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f4cbc06edd0 con 0x7f4cbc06c300 2023-12-16T21:43:46.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 --> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4cd00aff80 con 0x7f4cbc06c300 2023-12-16T21:43:46.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cb77fe640 1 -- 172.21.15.46:0/2432532883 <== osd.3 v2:172.21.15.46:6809/2268558465 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f4cc80a5880 con 0x7f4cbc06c300 2023-12-16T21:43:46.794 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:46.794 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:46.794 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7f4cbc06c300 msgr2=0x7f4cbc06e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:6809/2268558465,v1:172.21.15.46:6811/2268558465] conn(0x7f4cbc06c300 0x7f4cbc06e720 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4cbc0664a0 msgr2=0x7f4cbc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4cbc0664a0 0x7f4cbc068960 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0001380 tx=0x7f4cc0009040 comp rx=0 tx=0).stop 2023-12-16T21:43:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 msgr2=0x7f4cd01deea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 --2- 172.21.15.46:0/2432532883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cd0147120 0x7f4cd01deea0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400e8d0 tx=0x7f4cc400ed90 comp rx=0 tx=0).stop 2023-12-16T21:43:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4ccf7fe640 1 -- 172.21.15.46:0/2432532883 reap_dead start 2023-12-16T21:43:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 shutdown_connections 2023-12-16T21:43:46.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 >> 172.21.15.46:0/2432532883 conn(0x7f4cd00b4890 msgr2=0x7f4cd00b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 shutdown_connections 2023-12-16T21:43:46.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.791+0000 7f4cd5774640 1 -- 172.21.15.46:0/2432532883 wait complete. 2023-12-16T21:43:46.808 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-16T21:43:46.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac7f16640 1 Processor -- start 2023-12-16T21:43:46.922 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac7f16640 1 -- start start 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac7f16640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f4ac006a7e0 0x7f4ac01479c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac7f16640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ac0141c80 0x7f4ac01449c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac7f16640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac00701a0 con 0x7f4ac0147f00 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac7f16640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac006abc0 con 0x7f4ac0141c80 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac7f16640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac01484d0 con 0x7f4ac006a7e0 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac64b3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ac0141c80 0x7f4ac01449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac64b3640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f4ac0141c80 0x7f4ac01449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35692/0 (socket says 172.21.15.46:35692) 2023-12-16T21:43:46.924 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3152777621 learned_addr learned my addr 172.21.15.46:0/3152777621 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac5cb2640 1 --2- 172.21.15.46:0/3152777621 >> v2:172.21.15.161:3300/0 conn(0x7f4ac006a7e0 0x7f4ac01479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3152777621 >> v2:172.21.15.161:3300/0 conn(0x7f4ac006a7e0 msgr2=0x7f4ac01479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:46.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac64b3640 1 --2- 172.21.15.46:0/3152777621 >> v2:172.21.15.161:3300/0 conn(0x7f4ac006a7e0 0x7f4ac01479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3152777621 >> v1:172.21.15.161:6789/0 conn(0x7f4ac0147f00 legacy=0x7f4ac0141570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:46.925 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.919+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3152777621 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac01487b0 con 0x7f4ac0141c80 2023-12-16T21:43:46.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac64b3640 1 --2- 172.21.15.46:0/3152777621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0141c80 0x7f4ac01449c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00b02a0 tx=0x7f4ab801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb0b24b8c50dc70 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac4cb0640 1 -- 172.21.15.46:0/3152777621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80ab600 con 0x7f4ac0141c80 2023-12-16T21:43:46.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac4cb0640 1 -- 172.21.15.46:0/3152777621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab80a7050 con 0x7f4ac0141c80 2023-12-16T21:43:46.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac4cb0640 1 -- 172.21.15.46:0/3152777621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80ab980 con 0x7f4ac0141c80 2023-12-16T21:43:46.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3152777621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0141c80 msgr2=0x7f4ac01449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3152777621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0141c80 0x7f4ac01449c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ac00b02a0 tx=0x7f4ab801c430 comp rx=0 tx=0).stop 2023-12-16T21:43:46.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3152777621 shutdown_connections 2023-12-16T21:43:46.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3152777621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0141c80 0x7f4ac01449c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3152777621 >> v2:172.21.15.161:3300/0 conn(0x7f4ac006a7e0 0x7f4ac01479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.927 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3152777621 >> 172.21.15.46:0/3152777621 conn(0x7f4ac00b92c0 msgr2=0x7f4ac00b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3152777621 shutdown_connections 2023-12-16T21:43:46.928 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3152777621 wait complete. 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 Processor -- start 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 -- start start 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.923+0000 7f4ac7f16640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ac006a7e0 0x7f4ac00b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0141c80 0x7f4ac01e2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0147f00 0x7f4ac01e2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac01e2cc0 con 0x7f4ac0147f00 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac013a1d0 con 0x7f4ac0141c80 2023-12-16T21:43:46.930 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac0138c70 con 0x7f4ac006a7e0 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0147f00 0x7f4ac01e2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0147f00 0x7f4ac01e2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35698/0 (socket says 172.21.15.46:35698) 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3942260513 learned_addr learned my addr 172.21.15.46:0/3942260513 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac5cb2640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ac006a7e0 0x7f4ac00b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac54b1640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0141c80 0x7f4ac01e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ac006a7e0 msgr2=0x7f4ac00b7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ac006a7e0 0x7f4ac00b7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0141c80 msgr2=0x7f4ac01e2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0141c80 0x7f4ac01e2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 -- 172.21.15.46:0/3942260513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab80094d0 con 0x7f4ac0147f00 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac64b3640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0147f00 0x7f4ac01e2780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4ab801c470 tx=0x7f4ab80191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ab67fc640 1 -- 172.21.15.46:0/3942260513 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80b4840 con 0x7f4ac0147f00 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ab67fc640 1 -- 172.21.15.46:0/3942260513 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab80b49c0 con 0x7f4ac0147f00 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac0138f50 con 0x7f4ac0147f00 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ab67fc640 1 -- 172.21.15.46:0/3942260513 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab80b4d70 con 0x7f4ac0147f00 2023-12-16T21:43:46.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac0139e30 con 0x7f4ac0147f00 2023-12-16T21:43:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac0138db0 con 0x7f4ac0147f00 2023-12-16T21:43:46.933 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ab67fc640 1 -- 172.21.15.46:0/3942260513 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f4ab80b2040 con 0x7f4ac0147f00 2023-12-16T21:43:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ab67fc640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4aa4066470 0x7f4aa4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ab67fc640 1 -- 172.21.15.46:0/3942260513 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4ab811e100 con 0x7f4ac0147f00 2023-12-16T21:43:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac5cb2640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4aa4066470 0x7f4aa4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:46.934 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.927+0000 7f4ac5cb2640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4aa4066470 0x7f4aa4068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4abc003fd0 tx=0x7f4abc0039a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:46.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:46.931+0000 7f4ab67fc640 1 -- 172.21.15.46:0/3942260513 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f4ac0138db0 con 0x7f4ac0147f00 2023-12-16T21:43:47.090 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.087+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ac01390e0 con 0x7f4aa4066470 2023-12-16T21:43:47.091 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.087+0000 7f4ab67fc640 1 -- 172.21.15.46:0/3942260513 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4abc004780 con 0x7f4aa4066470 2023-12-16T21:43:47.092 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 3 to deep-scrub 2023-12-16T21:43:47.093 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4aa4066470 msgr2=0x7f4aa4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4aa4066470 0x7f4aa4068930 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4abc003fd0 tx=0x7f4abc0039a0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0147f00 msgr2=0x7f4ac01e2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0147f00 0x7f4ac01e2780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4ab801c470 tx=0x7f4ab80191d0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 shutdown_connections 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4aa4066470 0x7f4aa4068930 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4ac0147f00 0x7f4ac01e2780 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f4ac0141c80 0x7f4ac01e2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.094 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 --2- 172.21.15.46:0/3942260513 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f4ac006a7e0 0x7f4ac00b7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 >> 172.21.15.46:0/3942260513 conn(0x7f4ac00b92c0 msgr2=0x7f4ac0142850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 shutdown_connections 2023-12-16T21:43:47.095 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.091+0000 7f4ac7f16640 1 -- 172.21.15.46:0/3942260513 wait complete. 2023-12-16T21:43:47.110 INFO:tasks.ceph:Scrubbing osd.4 2023-12-16T21:43:47.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:47.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.235+0000 7f9845b67640 1 Processor -- start 2023-12-16T21:43:47.240 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.235+0000 7f9845b67640 1 -- start start 2023-12-16T21:43:47.242 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f9845b67640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 0x7f98400b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f9845b67640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f98400b0fb0 0x7f98400acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f9845b67640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9840072550 con 0x7f98400ae0f0 2023-12-16T21:43:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f9845b67640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98400ae750 con 0x7f98400b0fb0 2023-12-16T21:43:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f9845b67640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98400b2dd0 con 0x7f98400b29f0 2023-12-16T21:43:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983ffff640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f98400ae0f0 0x7f98400ad2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46512/0 (socket says 172.21.15.46:46512) 2023-12-16T21:43:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983ffff640 1 -- 172.21.15.46:0/1448710851 learned_addr learned my addr 172.21.15.46:0/1448710851 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:47.243 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 --2- 172.21.15.46:0/1448710851 >> v2:172.21.15.161:3300/0 conn(0x7f98400b0fb0 0x7f98400acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983f7fe640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 0x7f98400b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122418627 0 0) 0x7f98380014a0 con 0x7f98400ae0f0 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9828003530 con 0x7f98400ae0f0 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1082037407 0 0) 0x7f9828003530 con 0x7f98400ae0f0 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 msgr2=0x7f98400b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 0x7f98400b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 >> v2:172.21.15.161:3300/0 conn(0x7f98400b0fb0 msgr2=0x7f98400acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 --2- 172.21.15.46:0/1448710851 >> v2:172.21.15.161:3300/0 conn(0x7f98400b0fb0 0x7f98400acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.244 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400b0300 con 0x7f98400ae0f0 2023-12-16T21:43:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f9838001b90 con 0x7f98400ae0f0 2023-12-16T21:43:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 >> v1:172.21.15.46:6789/0 conn(0x7f98400ae0f0 legacy=0x7f98400ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 0x7f9828003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f98280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9828004d10 0x7f9828005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.245 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98400ae750 con 0x7f9828003f00 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98400b2dd0 con 0x7f9828004d10 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f982800ba70 con 0x7f98400b29f0 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f98280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983f7fe640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 0x7f9828003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983ffff640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9828004d10 0x7f9828005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 -- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 msgr2=0x7f9828003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f98400b29f0 0x7f9828003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 -- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9828004d10 msgr2=0x7f9828005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.246 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9828004d10 0x7f9828005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.247 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 -- 172.21.15.46:0/1448710851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9828003530 con 0x7f9828003f00 2023-12-16T21:43:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983effd640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f98280047d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9830009ab0 tx=0x7f983001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4104d0e6462b46 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300a5520 con 0x7f9828003f00 2023-12-16T21:43:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98300a5680 con 0x7f9828003f00 2023-12-16T21:43:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.239+0000 7f983e7fc640 1 -- 172.21.15.46:0/1448710851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98300a5a30 con 0x7f9828003f00 2023-12-16T21:43:47.250 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- 172.21.15.46:0/1448710851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 msgr2=0x7f98280047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 --2- 172.21.15.46:0/1448710851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f98280047d0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9830009ab0 tx=0x7f983001ca40 comp rx=0 tx=0).stop 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983ffff640 1 -- 172.21.15.46:0/1448710851 reap_dead start 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- 172.21.15.46:0/1448710851 shutdown_connections 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- 172.21.15.46:0/1448710851 >> 172.21.15.46:0/1448710851 conn(0x7f98400b4890 msgr2=0x7f98400b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- 172.21.15.46:0/1448710851 shutdown_connections 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- 172.21.15.46:0/1448710851 wait complete. 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 Processor -- start 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- start start 2023-12-16T21:43:47.251 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f984014b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98401e2690 0x7f98401e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9840141400 0x7f98401418b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9840142130 con 0x7f9828003f00 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984014c700 con 0x7f98401e2690 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984014cb20 con 0x7f9840141400 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f984014b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f984014b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35710/0 (socket says 172.21.15.46:35710) 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 -- 172.21.15.46:0/1629556264 learned_addr learned my addr 172.21.15.46:0/1629556264 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:47.252 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983effd640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98401e2690 0x7f98401e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983ffff640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9840141400 0x7f98401418b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 -- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9840141400 msgr2=0x7f98401418b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9840141400 0x7f98401418b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 -- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98401e2690 msgr2=0x7f98401e2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98401e2690 0x7f98401e2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 -- 172.21.15.46:0/1629556264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98400af000 con 0x7f9828003f00 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983effd640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f98401e2690 0x7f98401e2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:47.253 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983ffff640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9840141400 0x7f98401418b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f983f7fe640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f984014b4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9834011610 tx=0x7f9834011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834015870 con 0x7f9828003f00 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98340159d0 con 0x7f9828003f00 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9834015d40 con 0x7f9828003f00 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984014af00 con 0x7f9828003f00 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.243+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9840141df0 con 0x7f9828003f00 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f98400aee80 con 0x7f9828003f00 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f9834027470 con 0x7f9828003f00 2023-12-16T21:43:47.254 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f9827fff640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f982c0664a0 0x7f982c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9834084300 con 0x7f9828003f00 2023-12-16T21:43:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f9827fff640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f982c06c300 0x7f982c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 --> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f982c06edd0 con 0x7f982c06c300 2023-12-16T21:43:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9834084670 con 0x7f9828003f00 2023-12-16T21:43:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f983effd640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f982c0664a0 0x7f982c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.255 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f983ffff640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f982c06c300 0x7f982c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f983ffff640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f982c06c300 0x7f982c06e720 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.247+0000 7f983effd640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f982c0664a0 0x7f982c068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f98300af000 tx=0x7f98300046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.256 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.251+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== osd.4 v2:172.21.15.161:6800/3753525888 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f982c06edd0 con 0x7f982c06c300 2023-12-16T21:43:47.265 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 --> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f984006cbd0 con 0x7f982c06c300 2023-12-16T21:43:47.266 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9827fff640 1 -- 172.21.15.46:0/1629556264 <== osd.4 v2:172.21.15.161:6800/3753525888 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f983800d040 con 0x7f982c06c300 2023-12-16T21:43:47.266 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:47.266 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f982c06c300 msgr2=0x7f982c06e720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6800/3753525888,v1:172.21.15.161:6801/3753525888] conn(0x7f982c06c300 0x7f982c06e720 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.267 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f982c0664a0 msgr2=0x7f982c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f982c0664a0 0x7f982c068960 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f98300af000 tx=0x7f98300046d0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 msgr2=0x7f984014b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 --2- 172.21.15.46:0/1629556264 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9828003f00 0x7f984014b4c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9834011610 tx=0x7f9834011ad0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f983ffff640 1 -- 172.21.15.46:0/1629556264 reap_dead start 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 shutdown_connections 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 >> 172.21.15.46:0/1629556264 conn(0x7f98400b4890 msgr2=0x7f98400b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 shutdown_connections 2023-12-16T21:43:47.268 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.263+0000 7f9845b67640 1 -- 172.21.15.46:0/1629556264 wait complete. 2023-12-16T21:43:47.281 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-16T21:43:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d704b6640 1 Processor -- start 2023-12-16T21:43:47.394 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d704b6640 1 -- start start 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d704b6640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f1d680722c0 0x7f1d68148030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d704b6640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d680af8d0 0x7f1d68148570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d704b6640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d680b1800 con 0x7f1d680af4f0 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d704b6640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d680b1610 con 0x7f1d680af8d0 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d704b6640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d680b1aa0 con 0x7f1d680722c0 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6ea53640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d680af8d0 0x7f1d68148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6ea53640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f1d680af8d0 0x7f1d68148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35718/0 (socket says 172.21.15.46:35718) 2023-12-16T21:43:47.396 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6ea53640 1 -- 172.21.15.46:0/223592288 learned_addr learned my addr 172.21.15.46:0/223592288 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6da51640 1 --2- 172.21.15.46:0/223592288 >> v2:172.21.15.161:3300/0 conn(0x7f1d680722c0 0x7f1d68148030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6ea53640 1 -- 172.21.15.46:0/223592288 >> v2:172.21.15.161:3300/0 conn(0x7f1d680722c0 msgr2=0x7f1d68148030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6ea53640 1 --2- 172.21.15.46:0/223592288 >> v2:172.21.15.161:3300/0 conn(0x7f1d680722c0 0x7f1d68148030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6ea53640 1 -- 172.21.15.46:0/223592288 >> v1:172.21.15.161:6789/0 conn(0x7f1d680af4f0 legacy=0x7f1d6809caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6ea53640 1 -- 172.21.15.46:0/223592288 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68147b10 con 0x7f1d680af8d0 2023-12-16T21:43:47.397 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.391+0000 7f1d6da51640 1 --2- 172.21.15.46:0/223592288 >> v2:172.21.15.161:3300/0 conn(0x7f1d680722c0 0x7f1d68148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6ea53640 1 --2- 172.21.15.46:0/223592288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d68148570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1d680af120 tx=0x7f1d6001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709b01635e860d03 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6d250640 1 -- 172.21.15.46:0/223592288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d600a64c0 con 0x7f1d680af8d0 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6d250640 1 -- 172.21.15.46:0/223592288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d600a6620 con 0x7f1d680af8d0 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6d250640 1 -- 172.21.15.46:0/223592288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d600a69b0 con 0x7f1d680af8d0 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- 172.21.15.46:0/223592288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 msgr2=0x7f1d68148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 --2- 172.21.15.46:0/223592288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d68148570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1d680af120 tx=0x7f1d6001c900 comp rx=0 tx=0).stop 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- 172.21.15.46:0/223592288 shutdown_connections 2023-12-16T21:43:47.399 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 --2- 172.21.15.46:0/223592288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d68148570 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 --2- 172.21.15.46:0/223592288 >> v2:172.21.15.161:3300/0 conn(0x7f1d680722c0 0x7f1d68148030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- 172.21.15.46:0/223592288 >> 172.21.15.46:0/223592288 conn(0x7f1d6806a580 msgr2=0x7f1d6806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- 172.21.15.46:0/223592288 shutdown_connections 2023-12-16T21:43:47.400 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- 172.21.15.46:0/223592288 wait complete. 2023-12-16T21:43:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 Processor -- start 2023-12-16T21:43:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- start start 2023-12-16T21:43:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d680722c0 0x7f1d681e0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.402 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d680af4f0 0x7f1d6809a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d6809a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6809c8e0 con 0x7f1d680af8d0 2023-12-16T21:43:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d681466f0 con 0x7f1d680af4f0 2023-12-16T21:43:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d704b6640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d68146990 con 0x7f1d680722c0 2023-12-16T21:43:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6ea53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d6809a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6ea53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d6809a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35722/0 (socket says 172.21.15.46:35722) 2023-12-16T21:43:47.403 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6ea53640 1 -- 172.21.15.46:0/651447802 learned_addr learned my addr 172.21.15.46:0/651447802 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.395+0000 7f1d6da51640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d680af4f0 0x7f1d6809a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6e252640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d680722c0 0x7f1d681e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6ea53640 1 -- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d680722c0 msgr2=0x7f1d681e0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6ea53640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d680722c0 0x7f1d681e0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6ea53640 1 -- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d680af4f0 msgr2=0x7f1d6809a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6ea53640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d680af4f0 0x7f1d6809a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6ea53640 1 -- 172.21.15.46:0/651447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d60009b30 con 0x7f1d680af8d0 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6ea53640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d6809a700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1d6001afb0 tx=0x7f1d600196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.404 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d5effd640 1 -- 172.21.15.46:0/651447802 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d600a6c20 con 0x7f1d680af8d0 2023-12-16T21:43:47.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d5effd640 1 -- 172.21.15.46:0/651447802 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d600a6d80 con 0x7f1d680af8d0 2023-12-16T21:43:47.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d68142730 con 0x7f1d680af8d0 2023-12-16T21:43:47.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d5effd640 1 -- 172.21.15.46:0/651447802 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d600af6c0 con 0x7f1d680af8d0 2023-12-16T21:43:47.405 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d68142350 con 0x7f1d680af8d0 2023-12-16T21:43:47.406 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d680679e0 con 0x7f1d680af8d0 2023-12-16T21:43:47.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d5effd640 1 -- 172.21.15.46:0/651447802 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f1d600af820 con 0x7f1d680af8d0 2023-12-16T21:43:47.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d5effd640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1d4c0664a0 0x7f1d4c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d5effd640 1 -- 172.21.15.46:0/651447802 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1d600aa070 con 0x7f1d680af8d0 2023-12-16T21:43:47.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6e252640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1d4c0664a0 0x7f1d4c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.399+0000 7f1d6e252640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1d4c0664a0 0x7f1d4c068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d64004e30 tx=0x7f1d64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.407 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.403+0000 7f1d5effd640 1 -- 172.21.15.46:0/651447802 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f1d600ebdc0 con 0x7f1d680af8d0 2023-12-16T21:43:47.563 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.559+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d680679e0 con 0x7f1d4c0664a0 2023-12-16T21:43:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.559+0000 7f1d5effd640 1 -- 172.21.15.46:0/651447802 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1d680679e0 con 0x7f1d4c0664a0 2023-12-16T21:43:47.564 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 4 to deep-scrub 2023-12-16T21:43:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1d4c0664a0 msgr2=0x7f1d4c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1d4c0664a0 0x7f1d4c068960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1d64004e30 tx=0x7f1d64009040 comp rx=0 tx=0).stop 2023-12-16T21:43:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 msgr2=0x7f1d6809a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.566 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d6809a700 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1d6001afb0 tx=0x7f1d600196a0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 shutdown_connections 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f1d4c0664a0 0x7f1d4c068960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d680af8d0 0x7f1d6809a700 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f1d680af4f0 0x7f1d6809a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 --2- 172.21.15.46:0/651447802 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f1d680722c0 0x7f1d681e0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 >> 172.21.15.46:0/651447802 conn(0x7f1d6806a580 msgr2=0x7f1d68137030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 shutdown_connections 2023-12-16T21:43:47.567 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.563+0000 7f1d704b6640 1 -- 172.21.15.46:0/651447802 wait complete. 2023-12-16T21:43:47.583 INFO:tasks.ceph:Scrubbing osd.5 2023-12-16T21:43:47.584 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.703+0000 7f7ba84c3640 1 Processor -- start 2023-12-16T21:43:47.708 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.703+0000 7f7ba84c3640 1 -- start start 2023-12-16T21:43:47.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba84c3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00bc870 0x7f7ba0146700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba84c3640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ba00b4a10 0x7f7ba00b4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba84c3640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba00b87e0 con 0x7f7ba0140480 2023-12-16T21:43:47.709 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba84c3640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba00b3870 con 0x7f7ba00b4a10 2023-12-16T21:43:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba84c3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba00bcc50 con 0x7f7ba00bc870 2023-12-16T21:43:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00bc870 0x7f7ba0146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba5a5e640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ba00b4a10 0x7f7ba00b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00bc870 0x7f7ba0146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:45568/0 (socket says 172.21.15.46:45568) 2023-12-16T21:43:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 -- 172.21.15.46:0/3163577915 learned_addr learned my addr 172.21.15.46:0/3163577915 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:47.710 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba5a5e640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f7ba00b4a10 0x7f7ba00b4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:48480/0 (socket says 172.21.15.46:48480) 2023-12-16T21:43:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 -- 172.21.15.46:0/3163577915 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00b4a10 msgr2=0x7f7ba00b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 --2- 172.21.15.46:0/3163577915 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00b4a10 0x7f7ba00b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 -- 172.21.15.46:0/3163577915 >> v1:172.21.15.161:6789/0 conn(0x7f7ba0140480 legacy=0x7f7ba01408f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 -- 172.21.15.46:0/3163577915 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba00b3b10 con 0x7f7ba00bc870 2023-12-16T21:43:47.711 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba5a5e640 1 --2- 172.21.15.46:0/3163577915 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00b4a10 0x7f7ba00b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba625f640 1 --2- 172.21.15.46:0/3163577915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00bc870 0x7f7ba0146700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009870 tx=0x7f7b9c01c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c660febd7a67f741 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba525d640 1 -- 172.21.15.46:0/3163577915 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0ab600 con 0x7f7ba00bc870 2023-12-16T21:43:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba525d640 1 -- 172.21.15.46:0/3163577915 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c0a7050 con 0x7f7ba00bc870 2023-12-16T21:43:47.712 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.707+0000 7f7ba525d640 1 -- 172.21.15.46:0/3163577915 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0ab980 con 0x7f7ba00bc870 2023-12-16T21:43:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- 172.21.15.46:0/3163577915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00bc870 msgr2=0x7f7ba0146700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 --2- 172.21.15.46:0/3163577915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00bc870 0x7f7ba0146700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c009870 tx=0x7f7b9c01c360 comp rx=0 tx=0).stop 2023-12-16T21:43:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- 172.21.15.46:0/3163577915 shutdown_connections 2023-12-16T21:43:47.713 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 --2- 172.21.15.46:0/3163577915 >> v2:172.21.15.161:3300/0 conn(0x7f7ba00b4a10 0x7f7ba00b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 --2- 172.21.15.46:0/3163577915 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00bc870 0x7f7ba0146700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- 172.21.15.46:0/3163577915 >> 172.21.15.46:0/3163577915 conn(0x7f7ba00723e0 msgr2=0x7f7ba00bbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- 172.21.15.46:0/3163577915 shutdown_connections 2023-12-16T21:43:47.714 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- 172.21.15.46:0/3163577915 wait complete. 2023-12-16T21:43:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 Processor -- start 2023-12-16T21:43:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- start start 2023-12-16T21:43:47.716 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00b4a10 0x7f7ba01411c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba00bc870 0x7f7ba0138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0140480 0x7f7ba013f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba013fc70 con 0x7f7ba0140480 2023-12-16T21:43:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0142bc0 con 0x7f7ba00bc870 2023-12-16T21:43:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba0143aa0 con 0x7f7ba00b4a10 2023-12-16T21:43:47.717 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0140480 0x7f7ba013f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0140480 0x7f7ba013f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35728/0 (socket says 172.21.15.46:35728) 2023-12-16T21:43:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 -- 172.21.15.46:0/1423741473 learned_addr learned my addr 172.21.15.46:0/1423741473 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba625f640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00b4a10 0x7f7ba01411c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.718 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba5a5e640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba00bc870 0x7f7ba0138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 -- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00b4a10 msgr2=0x7f7ba01411c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f7ba00b4a10 0x7f7ba01411c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 -- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba00bc870 msgr2=0x7f7ba0138470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f7ba00bc870 0x7f7ba0138470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 -- 172.21.15.46:0/1423741473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009420 con 0x7f7ba0140480 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba6a60640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0140480 0x7f7ba013f730 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c730 tx=0x7f7b9800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98006d60 con 0x7f7ba0140480 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b98003980 con 0x7f7ba0140480 2023-12-16T21:43:47.719 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c0094d0 con 0x7f7ba0140480 2023-12-16T21:43:47.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b98003d30 con 0x7f7ba0140480 2023-12-16T21:43:47.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.711+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba00b3070 con 0x7f7ba0140480 2023-12-16T21:43:47.720 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ba013f300 con 0x7f7ba0140480 2023-12-16T21:43:47.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f7b980050d0 con 0x7f7ba0140480 2023-12-16T21:43:47.721 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7b96ffd640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b84066470 0x7f7b84068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7b98083ff0 con 0x7f7ba0140480 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7b96ffd640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f7b8406c2d0 0x7f7b8406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 --> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b8406eda0 con 0x7f7b8406c2d0 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b98084380 con 0x7f7ba0140480 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7ba625f640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b84066470 0x7f7b84068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7ba5a5e640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f7b8406c2d0 0x7f7b8406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7ba625f640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b84066470 0x7f7b84068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ad000 tx=0x7f7b9c0a6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.715+0000 7f7ba5a5e640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f7b8406c2d0 0x7f7b8406e6f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.722 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.719+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== osd.5 v2:172.21.15.161:6808/1602903861 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7b8406eda0 con 0x7f7b8406c2d0 2023-12-16T21:43:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 --> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7ba00b87e0 con 0x7f7b8406c2d0 2023-12-16T21:43:47.735 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7b96ffd640 1 -- 172.21.15.46:0/1423741473 <== osd.5 v2:172.21.15.161:6808/1602903861 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7ba00b87e0 con 0x7f7b8406c2d0 2023-12-16T21:43:47.736 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:47.736 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f7b8406c2d0 msgr2=0x7f7b8406e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6808/1602903861,v1:172.21.15.161:6809/1602903861] conn(0x7f7b8406c2d0 0x7f7b8406e6f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b84066470 msgr2=0x7f7b84068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7b84066470 0x7f7b84068930 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0ad000 tx=0x7f7b9c0a6000 comp rx=0 tx=0).stop 2023-12-16T21:43:47.737 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0140480 msgr2=0x7f7ba013f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 --2- 172.21.15.46:0/1423741473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7ba0140480 0x7f7ba013f730 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7b9800c730 tx=0x7f7b9800cb30 comp rx=0 tx=0).stop 2023-12-16T21:43:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba6a60640 1 -- 172.21.15.46:0/1423741473 reap_dead start 2023-12-16T21:43:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 shutdown_connections 2023-12-16T21:43:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 >> 172.21.15.46:0/1423741473 conn(0x7f7ba00723e0 msgr2=0x7f7ba00b90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 shutdown_connections 2023-12-16T21:43:47.738 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.731+0000 7f7ba84c3640 1 -- 172.21.15.46:0/1423741473 wait complete. 2023-12-16T21:43:47.752 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-16T21:43:47.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8abdae3640 1 Processor -- start 2023-12-16T21:43:47.870 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8abdae3640 1 -- start start 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8abdae3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 0x7f8ab814f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8abdae3640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80af850 con 0x7f8ab8146be0 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8abdae3640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab80ad7f0 con 0x7f8ab80b3b00 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8abdae3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80b3ee0 con 0x7f8ab81477a0 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab6ffd640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8ab8146be0 0x7f8ab8147090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46526/0 (socket says 172.21.15.46:46526) 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab6ffd640 1 -- 172.21.15.46:0/2397197946 learned_addr learned my addr 172.21.15.46:0/2397197946 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab7fff640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 0x7f8ab814f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.873 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168460329 0 0) 0x7f8ab80af850 con 0x7f8ab8146be0 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa4003530 con 0x7f8ab8146be0 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2280633337 0 0) 0x7f8aa4003530 con 0x7f8ab8146be0 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 msgr2=0x7f8ab814f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 0x7f8ab814f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 >> v1:172.21.15.161:6789/0 conn(0x7f8ab80b3b00 legacy=0x7f8ab80ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab80afb30 con 0x7f8ab8146be0 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f8aa8001a20 con 0x7f8ab8146be0 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 >> v1:172.21.15.46:6789/0 conn(0x7f8ab8146be0 legacy=0x7f8ab8147090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.874 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa40047d0 0x7f8aa4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 0x7f8aa4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab80b3ee0 con 0x7f8aa40039b0 2023-12-16T21:43:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa400ba40 con 0x7f8aa40047d0 2023-12-16T21:43:47.875 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa400bc00 con 0x7f8ab81477a0 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab77fe640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab6ffd640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa40047d0 0x7f8aa4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab7fff640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 0x7f8aa4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab77fe640 1 -- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 msgr2=0x7f8aa4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab77fe640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab81477a0 0x7f8aa4005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab77fe640 1 -- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa40047d0 msgr2=0x7f8aa4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab77fe640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8aa40047d0 0x7f8aa4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.876 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.867+0000 7f8ab77fe640 1 -- 172.21.15.46:0/2397197946 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa4003530 con 0x7f8aa40039b0 2023-12-16T21:43:47.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8ab77fe640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0001750 tx=0x7f8aa001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f3b04c15a99c3a server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.879 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa00052d0 con 0x7f8aa40039b0 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa00a7050 con 0x7f8aa40039b0 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8ab67fc640 1 -- 172.21.15.46:0/2397197946 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0003600 con 0x7f8aa40039b0 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 -- 172.21.15.46:0/2397197946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8aa40039b0 msgr2=0x7f8aa4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 --2- 172.21.15.46:0/2397197946 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8aa40039b0 0x7f8aa4003dd0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0001750 tx=0x7f8aa001c3a0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8ab7fff640 1 -- 172.21.15.46:0/2397197946 reap_dead start 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 -- 172.21.15.46:0/2397197946 shutdown_connections 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 -- 172.21.15.46:0/2397197946 >> 172.21.15.46:0/2397197946 conn(0x7f8ab806a580 msgr2=0x7f8ab806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 -- 172.21.15.46:0/2397197946 shutdown_connections 2023-12-16T21:43:47.880 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 -- 172.21.15.46:0/2397197946 wait complete. 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 Processor -- start 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.871+0000 7f8abdae3640 1 -- start start 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ab814c820 0x7f8ab81e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ab81e30c0 0x7f8ab8144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab8145190 0x7f8ab81dee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81e2410 con 0x7f8ab814c820 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81491a0 con 0x7f8ab81e30c0 2023-12-16T21:43:47.881 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab814cc00 con 0x7f8ab8145190 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ab814c820 0x7f8ab81e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ab814c820 0x7f8ab81e2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35750/0 (socket says 172.21.15.46:35750) 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 -- 172.21.15.46:0/612758692 learned_addr learned my addr 172.21.15.46:0/612758692 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab6ffd640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ab81e30c0 0x7f8ab8144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab7fff640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab8145190 0x7f8ab81dee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 -- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab8145190 msgr2=0x7f8ab81dee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab8145190 0x7f8ab81dee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 -- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ab81e30c0 msgr2=0x7f8ab8144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:47.882 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ab81e30c0 0x7f8ab8144c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 -- 172.21.15.46:0/612758692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab814ae20 con 0x7f8ab814c820 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab6ffd640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ab81e30c0 0x7f8ab8144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab77fe640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ab814c820 0x7f8ab81e2b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005040 tx=0x7f8aa4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8aaffff640 1 -- 172.21.15.46:0/612758692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0003800 con 0x7f8ab814c820 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8aaffff640 1 -- 172.21.15.46:0/612758692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa0003980 con 0x7f8ab814c820 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab81e2660 con 0x7f8ab814c820 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8aaffff640 1 -- 172.21.15.46:0/612758692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0003d30 con 0x7f8ab814c820 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab8070430 con 0x7f8ab814c820 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab80679e0 con 0x7f8ab814c820 2023-12-16T21:43:47.883 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8aaffff640 1 -- 172.21.15.46:0/612758692 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8aa00c1460 con 0x7f8ab814c820 2023-12-16T21:43:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8aaffff640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8a980664a0 0x7f8a98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8aaffff640 1 -- 172.21.15.46:0/612758692 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8aa011d8c0 con 0x7f8ab814c820 2023-12-16T21:43:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.875+0000 7f8ab6ffd640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8a980664a0 0x7f8a98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:47.884 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.879+0000 7f8ab6ffd640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8a980664a0 0x7f8a98068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80015b0 tx=0x7f8aa8001500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:47.885 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:47.879+0000 7f8aaffff640 1 -- 172.21.15.46:0/612758692 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8aa00eb030 con 0x7f8ab814c820 2023-12-16T21:43:48.031 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.027+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ab80ae280 con 0x7f8a980664a0 2023-12-16T21:43:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.027+0000 7f8aaffff640 1 -- 172.21.15.46:0/612758692 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8ab80ae280 con 0x7f8a980664a0 2023-12-16T21:43:48.032 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 5 to deep-scrub 2023-12-16T21:43:48.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8a980664a0 msgr2=0x7f8a98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8a980664a0 0x7f8a98068960 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80015b0 tx=0x7f8aa8001500 comp rx=0 tx=0).stop 2023-12-16T21:43:48.034 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ab814c820 msgr2=0x7f8ab81e2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ab814c820 0x7f8ab81e2b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005040 tx=0x7f8aa4005800 comp rx=0 tx=0).stop 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 shutdown_connections 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8a980664a0 0x7f8a98068960 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8ab8145190 0x7f8ab81dee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8ab81e30c0 0x7f8ab8144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 --2- 172.21.15.46:0/612758692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ab814c820 0x7f8ab81e2b80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 >> 172.21.15.46:0/612758692 conn(0x7f8ab806a580 msgr2=0x7f8ab806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 shutdown_connections 2023-12-16T21:43:48.035 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.031+0000 7f8abdae3640 1 -- 172.21.15.46:0/612758692 wait complete. 2023-12-16T21:43:48.050 INFO:tasks.ceph:Scrubbing osd.6 2023-12-16T21:43:48.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:48.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f560d4aa640 1 Processor -- start 2023-12-16T21:43:48.170 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f560d4aa640 1 -- start start 2023-12-16T21:43:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f560d4aa640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f560806c920 0x7f56080aff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f560d4aa640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f56080afa00 0x7f5608144a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f560d4aa640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56081453d0 con 0x7f5608144fa0 2023-12-16T21:43:48.172 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f560d4aa640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56080b3cd0 con 0x7f560806c920 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f560d4aa640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56080b1ea0 con 0x7f56080afa00 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f5608144fa0 0x7f56080b2790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:46538/0 (socket says 172.21.15.46:46538) 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 -- 172.21.15.46:0/1656452788 learned_addr learned my addr 172.21.15.46:0/1656452788 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5606ffd640 1 --2- 172.21.15.46:0/1656452788 >> v2:172.21.15.46:3300/0 conn(0x7f560806c920 0x7f56080aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56067fc640 1 --2- 172.21.15.46:0/1656452788 >> v2:172.21.15.161:3300/0 conn(0x7f56080afa00 0x7f5608144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963654091 0 0) 0x7f56081453d0 con 0x7f5608144fa0 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f0003530 con 0x7f5608144fa0 2023-12-16T21:43:48.173 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3195535024 0 0) 0x7f55f0003530 con 0x7f5608144fa0 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 >> v2:172.21.15.161:3300/0 conn(0x7f56080afa00 msgr2=0x7f5608144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 --2- 172.21.15.46:0/1656452788 >> v2:172.21.15.161:3300/0 conn(0x7f56080afa00 0x7f5608144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 >> v2:172.21.15.46:3300/0 conn(0x7f560806c920 msgr2=0x7f56080aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 --2- 172.21.15.46:0/1656452788 >> v2:172.21.15.46:3300/0 conn(0x7f560806c920 0x7f56080aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56080b1b70 con 0x7f5608144fa0 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f5600001a00 con 0x7f5608144fa0 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5606ffd640 1 --2- 172.21.15.46:0/1656452788 >> v2:172.21.15.46:3300/0 conn(0x7f560806c920 0x7f56080aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:48.174 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 >> v1:172.21.15.46:6789/0 conn(0x7f5608144fa0 legacy=0x7f56080b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f0003980 0x7f55f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f00047d0 0x7f55f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f55f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080b3cd0 con 0x7f55f0005190 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080b1ea0 con 0x7f55f00047d0 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f000bf20 con 0x7f55f0003980 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f55f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56067fc640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f00047d0 0x7f55f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.175 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5606ffd640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f0003980 0x7f55f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 -- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f0003980 msgr2=0x7f55f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f0003980 0x7f55f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 -- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f00047d0 msgr2=0x7f55f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f55f00047d0 0x7f55f0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f56077fe640 1 -- 172.21.15.46:0/1656452788 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0003530 con 0x7f55f0005190 2023-12-16T21:43:48.176 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.167+0000 7f5606ffd640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f55f0003980 0x7f55f0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f56077fe640 1 -- 172.21.15.46:0/1656452788 reap_dead start 2023-12-16T21:43:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f56077fe640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f55f000b990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f560000a840 tx=0x7f560001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260fff77b05aa962 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56000ac620 con 0x7f55f0005190 2023-12-16T21:43:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56000a8070 con 0x7f55f0005190 2023-12-16T21:43:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f5605ffb640 1 -- 172.21.15.46:0/1656452788 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56000ac9e0 con 0x7f55f0005190 2023-12-16T21:43:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 -- 172.21.15.46:0/1656452788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 msgr2=0x7f55f000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.180 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f55f000b990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f560000a840 tx=0x7f560001d340 comp rx=0 tx=0).stop 2023-12-16T21:43:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 -- 172.21.15.46:0/1656452788 shutdown_connections 2023-12-16T21:43:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 --2- 172.21.15.46:0/1656452788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f55f000b990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 -- 172.21.15.46:0/1656452788 >> 172.21.15.46:0/1656452788 conn(0x7f56080b4890 msgr2=0x7f56080b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 -- 172.21.15.46:0/1656452788 shutdown_connections 2023-12-16T21:43:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 -- 172.21.15.46:0/1656452788 wait complete. 2023-12-16T21:43:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 Processor -- start 2023-12-16T21:43:48.181 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.171+0000 7f560d4aa640 1 -- start start 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f56081ce990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56080acdb0 0x7f56080ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56080adb90 0x7f56081e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5608140140 con 0x7f55f0005190 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5608141e30 con 0x7f56080acdb0 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56080ae190 con 0x7f56080adb90 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f56081ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f56081ce990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35784/0 (socket says 172.21.15.46:35784) 2023-12-16T21:43:48.182 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 -- 172.21.15.46:0/2241906654 learned_addr learned my addr 172.21.15.46:0/2241906654 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f56067fc640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56080acdb0 0x7f56080ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f56077fe640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56080adb90 0x7f56081e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 -- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56080adb90 msgr2=0x7f56081e2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f56080adb90 0x7f56081e2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 -- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56080acdb0 msgr2=0x7f56080ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f56080acdb0 0x7f56080ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 -- 172.21.15.46:0/2241906654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560813ed70 con 0x7f55f0005190 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f5606ffd640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f56081ce990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f55f8010920 tx=0x7f55f8010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.183 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f800ec20 con 0x7f55f0005190 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f800eda0 con 0x7f55f0005190 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560813f050 con 0x7f55f0005190 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f801c640 con 0x7f55f0005190 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56081cb100 con 0x7f55f0005190 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f56080b3f20 con 0x7f55f0005190 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f55f801c7a0 con 0x7f55f0005190 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f55f4066470 0x7f55f4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.184 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f55f80843a0 con 0x7f55f0005190 2023-12-16T21:43:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7f55f406c2d0 0x7f55f406e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 --> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f55f406eda0 con 0x7f55f406c2d0 2023-12-16T21:43:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55f80847f0 con 0x7f55f0005190 2023-12-16T21:43:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f56067fc640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f55f4066470 0x7f55f4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.185 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.175+0000 7f56077fe640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7f55f406c2d0 0x7f55f406e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.179+0000 7f56067fc640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f55f4066470 0x7f55f4068930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5608071a20 tx=0x7f55fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.179+0000 7f56077fe640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7f55f406c2d0 0x7f55f406e6f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.186 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.179+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== osd.6 v2:172.21.15.161:6824/2291774553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f55f406eda0 con 0x7f55f406c2d0 2023-12-16T21:43:48.195 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.191+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 --> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f56080689d0 con 0x7f55f406c2d0 2023-12-16T21:43:48.196 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.191+0000 7f55ef7fe640 1 -- 172.21.15.46:0/2241906654 <== osd.6 v2:172.21.15.161:6824/2291774553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f56080689d0 con 0x7f55f406c2d0 2023-12-16T21:43:48.196 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:48.197 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:48.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7f55f406c2d0 msgr2=0x7f55f406e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6824/2291774553,v1:172.21.15.161:6825/2291774553] conn(0x7f55f406c2d0 0x7f55f406e6f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f55f4066470 msgr2=0x7f55f4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f55f4066470 0x7f55f4068930 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5608071a20 tx=0x7f55fc008040 comp rx=0 tx=0).stop 2023-12-16T21:43:48.198 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 msgr2=0x7f56081ce990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 --2- 172.21.15.46:0/2241906654 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55f0005190 0x7f56081ce990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f55f8010920 tx=0x7f55f8010de0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f56077fe640 1 -- 172.21.15.46:0/2241906654 reap_dead start 2023-12-16T21:43:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 shutdown_connections 2023-12-16T21:43:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 >> 172.21.15.46:0/2241906654 conn(0x7f56080b4890 msgr2=0x7f56080afef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 shutdown_connections 2023-12-16T21:43:48.199 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.195+0000 7f560d4aa640 1 -- 172.21.15.46:0/2241906654 wait complete. 2023-12-16T21:43:48.212 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-16T21:43:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27bd18d640 1 Processor -- start 2023-12-16T21:43:48.319 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27bd18d640 1 -- start start 2023-12-16T21:43:48.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27bd18d640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f27b80b4b80 0x7f27b8144a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27bd18d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b8144f40 0x7f27b814ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.320 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27bd18d640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b80b0560 con 0x7f27b813e890 2023-12-16T21:43:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27bd18d640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b8073b60 con 0x7f27b80b4b80 2023-12-16T21:43:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27bd18d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80b2bd0 con 0x7f27b8144f40 2023-12-16T21:43:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b759e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b8144f40 0x7f27b814ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b659c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f27b80b4b80 0x7f27b8144a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b759e640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b8144f40 0x7f27b814ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:45646/0 (socket says 172.21.15.46:45646) 2023-12-16T21:43:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b659c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f27b80b4b80 0x7f27b8144a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3300/0 says I am v2:172.21.15.46:48568/0 (socket says 172.21.15.46:48568) 2023-12-16T21:43:48.321 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b759e640 1 -- 172.21.15.46:0/3643673972 learned_addr learned my addr 172.21.15.46:0/3643673972 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b659c640 1 -- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b8144f40 msgr2=0x7f27b814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b659c640 1 --2- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b8144f40 0x7f27b814ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b659c640 1 -- 172.21.15.46:0/3643673972 >> v1:172.21.15.161:6789/0 conn(0x7f27b813e890 legacy=0x7f27b8073450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b659c640 1 -- 172.21.15.46:0/3643673972 --> v2:172.21.15.161:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8145390 con 0x7f27b80b4b80 2023-12-16T21:43:48.322 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.315+0000 7f27b759e640 1 --2- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b8144f40 0x7f27b814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-16T21:43:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27b659c640 1 --2- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 0x7f27b8144a00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009d90 tx=0x7f27a801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c921c0c6697d641 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27b5d9b640 1 -- 172.21.15.46:0/3643673972 <== mon.1 v2:172.21.15.161:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a80a5780 con 0x7f27b80b4b80 2023-12-16T21:43:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27b5d9b640 1 -- 172.21.15.46:0/3643673972 <== mon.1 v2:172.21.15.161:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a80a58e0 con 0x7f27b80b4b80 2023-12-16T21:43:48.323 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27b5d9b640 1 -- 172.21.15.46:0/3643673972 <== mon.1 v2:172.21.15.161:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a80a5c70 con 0x7f27b80b4b80 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 -- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 msgr2=0x7f27b8144a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 --2- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 0x7f27b8144a00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f27a8009d90 tx=0x7f27a801c8c0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 -- 172.21.15.46:0/3643673972 shutdown_connections 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 --2- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b8144f40 0x7f27b814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 --2- 172.21.15.46:0/3643673972 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 0x7f27b8144a00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 -- 172.21.15.46:0/3643673972 >> 172.21.15.46:0/3643673972 conn(0x7f27b806c050 msgr2=0x7f27b806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 -- 172.21.15.46:0/3643673972 shutdown_connections 2023-12-16T21:43:48.325 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 -- 172.21.15.46:0/3643673972 wait complete. 2023-12-16T21:43:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.319+0000 7f27bd18d640 1 Processor -- start 2023-12-16T21:43:48.326 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 -- start start 2023-12-16T21:43:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 0x7f27b8147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b813e890 0x7f27b806ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.328 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b8144f40 0x7f27b8070500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8146180 con 0x7f27b8144f40 2023-12-16T21:43:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8147490 con 0x7f27b80b4b80 2023-12-16T21:43:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8146380 con 0x7f27b813e890 2023-12-16T21:43:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b8144f40 0x7f27b8070500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b8144f40 0x7f27b8070500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35796/0 (socket says 172.21.15.46:35796) 2023-12-16T21:43:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 -- 172.21.15.46:0/65848850 learned_addr learned my addr 172.21.15.46:0/65848850 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:48.329 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b6d9d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 0x7f27b8147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b659c640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b813e890 0x7f27b806ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 -- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b813e890 msgr2=0x7f27b806ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b813e890 0x7f27b806ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 -- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 msgr2=0x7f27b8147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 0x7f27b8147ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.330 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 -- 172.21.15.46:0/65848850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b00094d0 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b759e640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b8144f40 0x7f27b8070500 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e930 tx=0x7f27b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27af7fe640 1 -- 172.21.15.46:0/65848850 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004020 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27af7fe640 1 -- 172.21.15.46:0/65848850 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b0004180 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a8009a70 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27af7fe640 1 -- 172.21.15.46:0/65848850 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004510 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b8148410 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b806d440 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27af7fe640 1 -- 172.21.15.46:0/65848850 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f27b000c940 con 0x7f27b8144f40 2023-12-16T21:43:48.331 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27af7fe640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2798066470 0x7f2798068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27af7fe640 1 -- 172.21.15.46:0/65848850 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f27b0083a30 con 0x7f27b8144f40 2023-12-16T21:43:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.323+0000 7f27b6d9d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2798066470 0x7f2798068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.327+0000 7f27b6d9d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2798066470 0x7f2798068930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27b814ed20 tx=0x7f27a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.332 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.327+0000 7f27af7fe640 1 -- 172.21.15.46:0/65848850 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f27b00511a0 con 0x7f27b8144f40 2023-12-16T21:43:48.480 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.475+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f27b81444b0 con 0x7f2798066470 2023-12-16T21:43:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27af7fe640 1 -- 172.21.15.46:0/65848850 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f27a00050d0 con 0x7f2798066470 2023-12-16T21:43:48.481 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 6 to deep-scrub 2023-12-16T21:43:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2798066470 msgr2=0x7f2798068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2798066470 0x7f2798068930 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f27b814ed20 tx=0x7f27a0009040 comp rx=0 tx=0).stop 2023-12-16T21:43:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b8144f40 msgr2=0x7f27b8070500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b8144f40 0x7f27b8070500 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f27b000e930 tx=0x7f27b000edf0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.483 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 shutdown_connections 2023-12-16T21:43:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f2798066470 0x7f2798068930 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b8144f40 0x7f27b8070500 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f27b813e890 0x7f27b806ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 --2- 172.21.15.46:0/65848850 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f27b80b4b80 0x7f27b8147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 >> 172.21.15.46:0/65848850 conn(0x7f27b806c050 msgr2=0x7f27b80b6510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.484 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 shutdown_connections 2023-12-16T21:43:48.485 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.479+0000 7f27bd18d640 1 -- 172.21.15.46:0/65848850 wait complete. 2023-12-16T21:43:48.498 INFO:tasks.ceph:Scrubbing osd.7 2023-12-16T21:43:48.498 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-16T21:43:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 Processor -- start 2023-12-16T21:43:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 -- start start 2023-12-16T21:43:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44540b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f44540b0fb0 0x7f44540acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.629 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44540ae0f0 0x7f44540ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44540ae750 con 0x7f44540ae0f0 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4454072550 con 0x7f44540b0fb0 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44594a4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44540b2dd0 con 0x7f44540b29f0 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44540ae0f0 0x7f44540ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f44540ae0f0 0x7f44540ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36876/0 (socket says 172.21.15.46:36876) 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 -- 172.21.15.46:0/1875807021 learned_addr learned my addr 172.21.15.46:0/1875807021 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f4452ffd640 1 --2- 172.21.15.46:0/1875807021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44540b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44527fc640 1 --2- 172.21.15.46:0/1875807021 >> v2:172.21.15.161:3300/0 conn(0x7f44540b0fb0 0x7f44540acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.630 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 -- 172.21.15.46:0/1875807021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 msgr2=0x7f44540b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 --2- 172.21.15.46:0/1875807021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44540b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 -- 172.21.15.46:0/1875807021 >> v2:172.21.15.161:3300/0 conn(0x7f44540b0fb0 msgr2=0x7f44540acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 --2- 172.21.15.46:0/1875807021 >> v2:172.21.15.161:3300/0 conn(0x7f44540b0fb0 0x7f44540acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 -- 172.21.15.46:0/1875807021 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540b0300 con 0x7f44540ae0f0 2023-12-16T21:43:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.623+0000 7f44537fe640 1 --2- 172.21.15.46:0/1875807021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44540ae0f0 0x7f44540ad2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f44540b47f0 tx=0x7f444c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653796f1b450e660 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f4451ffb640 1 -- 172.21.15.46:0/1875807021 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0a5560 con 0x7f44540ae0f0 2023-12-16T21:43:48.631 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f4451ffb640 1 -- 172.21.15.46:0/1875807021 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c0a56c0 con 0x7f44540ae0f0 2023-12-16T21:43:48.632 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f4451ffb640 1 -- 172.21.15.46:0/1875807021 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0a5a50 con 0x7f44540ae0f0 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- 172.21.15.46:0/1875807021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44540ae0f0 msgr2=0x7f44540ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 --2- 172.21.15.46:0/1875807021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44540ae0f0 0x7f44540ad2f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f44540b47f0 tx=0x7f444c01c8e0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- 172.21.15.46:0/1875807021 shutdown_connections 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 --2- 172.21.15.46:0/1875807021 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44540ae0f0 0x7f44540ad2f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 --2- 172.21.15.46:0/1875807021 >> v2:172.21.15.161:3300/0 conn(0x7f44540b0fb0 0x7f44540acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 --2- 172.21.15.46:0/1875807021 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44540b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- 172.21.15.46:0/1875807021 >> 172.21.15.46:0/1875807021 conn(0x7f44540b4890 msgr2=0x7f44540b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.635 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- 172.21.15.46:0/1875807021 shutdown_connections 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- 172.21.15.46:0/1875807021 wait complete. 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 Processor -- start 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- start start 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44540b0fb0 0x7f4454140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44541e24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44541e2a20 0x7f445414b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44541419d0 con 0x7f44541e2a20 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445413f8b0 con 0x7f44540b0fb0 2023-12-16T21:43:48.636 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44541e2e80 con 0x7f44540b29f0 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44541e2a20 0x7f445414b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44541e2a20 0x7f445414b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36888/0 (socket says 172.21.15.46:36888) 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 -- 172.21.15.46:0/2601879374 learned_addr learned my addr 172.21.15.46:0/2601879374 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44527fc640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44541e24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f4452ffd640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44540b0fb0 0x7f4454140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 -- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 msgr2=0x7f44541e24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44541e24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.637 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 -- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44540b0fb0 msgr2=0x7f4454140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44540b0fb0 0x7f4454140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 -- 172.21.15.46:0/2601879374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c009b30 con 0x7f44541e2a20 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f4452ffd640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f44540b0fb0 0x7f4454140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44527fc640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f44540b29f0 0x7f44541e24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44537fe640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44541e2a20 0x7f445414b800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f444c0af040 tx=0x7f444c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0b8670 con 0x7f44541e2a20 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444c0b87d0 con 0x7f44541e2a20 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44540af000 con 0x7f44541e2a20 2023-12-16T21:43:48.638 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444c0b8b80 con 0x7f44541e2a20 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.627+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445413bdb0 con 0x7f44541e2a20 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f44540aee80 con 0x7f44541e2a20 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f444c0b8ce0 con 0x7f44541e2a20 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f443b7fe640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4440066470 0x7f4440068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f444c11f700 con 0x7f44541e2a20 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f443b7fe640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7f444006c2d0 0x7f444006e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 --> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f444006eda0 con 0x7f444006c2d0 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f444c11fb50 con 0x7f44541e2a20 2023-12-16T21:43:48.639 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f4452ffd640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4440066470 0x7f4440068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f44527fc640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7f444006c2d0 0x7f444006e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f44527fc640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7f444006c2d0 0x7f444006e6f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.640 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.631+0000 7f4452ffd640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4440066470 0x7f4440068930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4444009720 tx=0x7f4444008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.641 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.635+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== osd.7 v2:172.21.15.161:6816/2824550480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f444006eda0 con 0x7f444006c2d0 2023-12-16T21:43:48.651 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 --> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f445406cbb0 con 0x7f444006c2d0 2023-12-16T21:43:48.652 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f443b7fe640 1 -- 172.21.15.46:0/2601879374 <== osd.7 v2:172.21.15.161:6816/2824550480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f445406cbb0 con 0x7f444006c2d0 2023-12-16T21:43:48.652 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2023-12-16T21:43:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-12-16T21:43:48.653 INFO:teuthology.orchestra.run.smithi046.stdout:} 2023-12-16T21:43:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7f444006c2d0 msgr2=0x7f444006e6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44594a4640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6816/2824550480,v1:172.21.15.161:6817/2824550480] conn(0x7f444006c2d0 0x7f444006e6f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4440066470 msgr2=0x7f4440068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44594a4640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f4440066470 0x7f4440068930 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4444009720 tx=0x7f4444008040 comp rx=0 tx=0).stop 2023-12-16T21:43:48.654 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44541e2a20 msgr2=0x7f445414b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44594a4640 1 --2- 172.21.15.46:0/2601879374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44541e2a20 0x7f445414b800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f444c0af040 tx=0x7f444c0196e0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.647+0000 7f44537fe640 1 -- 172.21.15.46:0/2601879374 reap_dead start 2023-12-16T21:43:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.651+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 shutdown_connections 2023-12-16T21:43:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.651+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 >> 172.21.15.46:0/2601879374 conn(0x7f44540b4890 msgr2=0x7f4454142750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.651+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 shutdown_connections 2023-12-16T21:43:48.655 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.651+0000 7f44594a4640 1 -- 172.21.15.46:0/2601879374 wait complete. 2023-12-16T21:43:48.670 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-16T21:43:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 Processor -- start 2023-12-16T21:43:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 -- start start 2023-12-16T21:43:48.792 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f0b280bc8f0 0x7f0b280b4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28146b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b281401d0 0x7f0b28140660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280b5220 con 0x7f0b281401d0 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b280bccd0 con 0x7f0b280bc8f0 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b30317640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b280b3c00 con 0x7f0b28146720 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b281401d0 0x7f0b28140660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f0b281401d0 0x7f0b28140660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36898/0 (socket says 172.21.15.46:36898) 2023-12-16T21:43:48.793 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 -- 172.21.15.46:0/2049595065 learned_addr learned my addr 172.21.15.46:0/2049595065 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e0b3640 1 --2- 172.21.15.46:0/2049595065 >> v2:172.21.15.161:3300/0 conn(0x7f0b280bc8f0 0x7f0b280b4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2d8b2640 1 --2- 172.21.15.46:0/2049595065 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28146b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 -- 172.21.15.46:0/2049595065 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 msgr2=0x7f0b28146b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 --2- 172.21.15.46:0/2049595065 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28146b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 -- 172.21.15.46:0/2049595065 >> v2:172.21.15.161:3300/0 conn(0x7f0b280bc8f0 msgr2=0x7f0b280b4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 --2- 172.21.15.46:0/2049595065 >> v2:172.21.15.161:3300/0 conn(0x7f0b280bc8f0 0x7f0b280b4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e8b4640 1 -- 172.21.15.46:0/2049595065 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b280b3ee0 con 0x7f0b281401d0 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2e0b3640 1 --2- 172.21.15.46:0/2049595065 >> v2:172.21.15.161:3300/0 conn(0x7f0b280bc8f0 0x7f0b280b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:48.794 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.787+0000 7f0b2d8b2640 1 --2- 172.21.15.46:0/2049595065 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:43:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b2e8b4640 1 --2- 172.21.15.46:0/2049595065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281401d0 0x7f0b28140660 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0b280b7b10 tx=0x7f0b2001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be62d7a21ced4cf4 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.795 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b2d0b1640 1 -- 172.21.15.46:0/2049595065 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b200ac410 con 0x7f0b281401d0 2023-12-16T21:43:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b2d0b1640 1 -- 172.21.15.46:0/2049595065 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b200ac570 con 0x7f0b281401d0 2023-12-16T21:43:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b2d0b1640 1 -- 172.21.15.46:0/2049595065 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b200ac900 con 0x7f0b281401d0 2023-12-16T21:43:48.796 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 -- 172.21.15.46:0/2049595065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281401d0 msgr2=0x7f0b28140660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 --2- 172.21.15.46:0/2049595065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281401d0 0x7f0b28140660 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0b280b7b10 tx=0x7f0b2001c500 comp rx=0 tx=0).stop 2023-12-16T21:43:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 -- 172.21.15.46:0/2049595065 shutdown_connections 2023-12-16T21:43:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 --2- 172.21.15.46:0/2049595065 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281401d0 0x7f0b28140660 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 --2- 172.21.15.46:0/2049595065 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28146b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 --2- 172.21.15.46:0/2049595065 >> v2:172.21.15.161:3300/0 conn(0x7f0b280bc8f0 0x7f0b280b4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 -- 172.21.15.46:0/2049595065 >> 172.21.15.46:0/2049595065 conn(0x7f0b280722b0 msgr2=0x7f0b280bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.797 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 -- 172.21.15.46:0/2049595065 shutdown_connections 2023-12-16T21:43:48.798 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.791+0000 7f0b30317640 1 -- 172.21.15.46:0/2049595065 wait complete. 2023-12-16T21:43:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 Processor -- start 2023-12-16T21:43:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 -- start start 2023-12-16T21:43:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b280bc8f0 0x7f0b2813c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b281401d0 0x7f0b28140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28139160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2814a8e0 con 0x7f0b280bc8f0 2023-12-16T21:43:48.800 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2814ab80 con 0x7f0b281401d0 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b28141500 con 0x7f0b28146720 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b280bc8f0 0x7f0b2813c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b280bc8f0 0x7f0b2813c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36904/0 (socket says 172.21.15.46:36904) 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 -- 172.21.15.46:0/2559497944 learned_addr learned my addr 172.21.15.46:0/2559497944 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2d8b2640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b281401d0 0x7f0b28140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 -- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 msgr2=0x7f0b28139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.801 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 -- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b281401d0 msgr2=0x7f0b28140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b281401d0 0x7f0b28140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 -- 172.21.15.46:0/2559497944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b200094d0 con 0x7f0b280bc8f0 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2d8b2640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b281401d0 0x7f0b28140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b2e0b3640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b280bc8f0 0x7f0b2813c8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009fc0 tx=0x7f0b2400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b1effd640 1 -- 172.21.15.46:0/2559497944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24003ba0 con 0x7f0b280bc8f0 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b1effd640 1 -- 172.21.15.46:0/2559497944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2400f040 con 0x7f0b280bc8f0 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b28143de0 con 0x7f0b280bc8f0 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b1effd640 1 -- 172.21.15.46:0/2559497944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b24004030 con 0x7f0b280bc8f0 2023-12-16T21:43:48.802 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b28143820 con 0x7f0b280bc8f0 2023-12-16T21:43:48.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.795+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b280689d0 con 0x7f0b280bc8f0 2023-12-16T21:43:48.803 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.799+0000 7f0b1effd640 1 -- 172.21.15.46:0/2559497944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f0b24004190 con 0x7f0b280bc8f0 2023-12-16T21:43:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.799+0000 7f0b1effd640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0b0c066470 0x7f0b0c068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.799+0000 7f0b1effd640 1 -- 172.21.15.46:0/2559497944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0b24083e70 con 0x7f0b280bc8f0 2023-12-16T21:43:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.799+0000 7f0b2d8b2640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0b0c066470 0x7f0b0c068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:48.804 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.799+0000 7f0b2d8b2640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0b0c066470 0x7f0b0c068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b2813d200 tx=0x7f0b18008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:48.805 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.803+0000 7f0b1effd640 1 -- 172.21.15.46:0/2559497944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f0b24051690 con 0x7f0b280bc8f0 2023-12-16T21:43:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.959+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b280b8740 con 0x7f0b0c066470 2023-12-16T21:43:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.959+0000 7f0b1effd640 1 -- 172.21.15.46:0/2559497944 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0b18015440 con 0x7f0b0c066470 2023-12-16T21:43:48.962 INFO:teuthology.orchestra.run.smithi046.stderr:instructed osd(s) 7 to deep-scrub 2023-12-16T21:43:48.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0b0c066470 msgr2=0x7f0b0c068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.966 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0b0c066470 0x7f0b0c068930 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0b2813d200 tx=0x7f0b18008040 comp rx=0 tx=0).stop 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b280bc8f0 msgr2=0x7f0b2813c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b280bc8f0 0x7f0b2813c8c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0b24009fc0 tx=0x7f0b2400d560 comp rx=0 tx=0).stop 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 shutdown_connections 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f0b0c066470 0x7f0b0c068930 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f0b28146720 0x7f0b28139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f0b281401d0 0x7f0b28140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 --2- 172.21.15.46:0/2559497944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b280bc8f0 0x7f0b2813c8c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 >> 172.21.15.46:0/2559497944 conn(0x7f0b280722b0 msgr2=0x7f0b28145e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:48.967 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 shutdown_connections 2023-12-16T21:43:48.968 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:48.963+0000 7f0b30317640 1 -- 172.21.15.46:0/2559497944 wait complete. 2023-12-16T21:43:48.982 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:43:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f739dc94640 1 Processor -- start 2023-12-16T21:43:49.105 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f739dc94640 1 -- start start 2023-12-16T21:43:49.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f739dc94640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f739814a390 0x7f73980b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:49.106 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f739dc94640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f739813ff50 0x7f73981403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f739dc94640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73980bcbe0 con 0x7f73980bc800 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f739dc94640 1 -- --> v2:172.21.15.46:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73980b38f0 con 0x7f739813ff50 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f739dc94640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73980b3bd0 con 0x7f739814a390 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f7397fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f739813ff50 0x7f73981403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f7397fff640 1 --2- >> v2:172.21.15.46:3300/0 conn(0x7f739813ff50 0x7f73981403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36910/0 (socket says 172.21.15.46:36910) 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f7397fff640 1 -- 172.21.15.46:0/38305700 learned_addr learned my addr 172.21.15.46:0/38305700 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f738ffff640 1 --2- 172.21.15.46:0/38305700 >> v2:172.21.15.161:3300/0 conn(0x7f739814a390 0x7f73980b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f7397fff640 1 -- 172.21.15.46:0/38305700 >> v2:172.21.15.161:3300/0 conn(0x7f739814a390 msgr2=0x7f73980b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:43:49.107 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f7397fff640 1 --2- 172.21.15.46:0/38305700 >> v2:172.21.15.161:3300/0 conn(0x7f739814a390 0x7f73980b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f7397fff640 1 -- 172.21.15.46:0/38305700 >> v1:172.21.15.161:6789/0 conn(0x7f73980bc800 legacy=0x7f7398149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:49.108 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.103+0000 7f7397fff640 1 -- 172.21.15.46:0/38305700 --> v2:172.21.15.46:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980b3eb0 con 0x7f739813ff50 2023-12-16T21:43:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7397fff640 1 --2- 172.21.15.46:0/38305700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73981403c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f739814ac70 tx=0x7f739001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed81d4721768c0f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7396ffd640 1 -- 172.21.15.46:0/38305700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73900a57a0 con 0x7f739813ff50 2023-12-16T21:43:49.109 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7396ffd640 1 -- 172.21.15.46:0/38305700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73900a5900 con 0x7f739813ff50 2023-12-16T21:43:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7396ffd640 1 -- 172.21.15.46:0/38305700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73900a5c90 con 0x7f739813ff50 2023-12-16T21:43:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- 172.21.15.46:0/38305700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 msgr2=0x7f73981403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:49.110 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 --2- 172.21.15.46:0/38305700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73981403c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f739814ac70 tx=0x7f739001c9d0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- 172.21.15.46:0/38305700 shutdown_connections 2023-12-16T21:43:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 --2- 172.21.15.46:0/38305700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73981403c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 --2- 172.21.15.46:0/38305700 >> v2:172.21.15.161:3300/0 conn(0x7f739814a390 0x7f73980b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- 172.21.15.46:0/38305700 >> 172.21.15.46:0/38305700 conn(0x7f73980722b0 msgr2=0x7f73980bbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- 172.21.15.46:0/38305700 shutdown_connections 2023-12-16T21:43:49.111 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- 172.21.15.46:0/38305700 wait complete. 2023-12-16T21:43:49.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 Processor -- start 2023-12-16T21:43:49.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- start start 2023-12-16T21:43:49.113 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73980bc800 0x7f7398140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73980bb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f739814a390 0x7f7398142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73981df8a0 con 0x7f739813ff50 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73981426a0 con 0x7f739814a390 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f739dc94640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7398143e10 con 0x7f73980bc800 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73980bb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73980bb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36920/0 (socket says 172.21.15.46:36920) 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 -- 172.21.15.46:0/455120019 learned_addr learned my addr 172.21.15.46:0/455120019 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:43:49.114 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f73977fe640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73980bc800 0x7f7398140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7397fff640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f739814a390 0x7f7398142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 -- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73980bc800 msgr2=0x7f7398140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73980bc800 0x7f7398140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 -- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f739814a390 msgr2=0x7f7398142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f739814a390 0x7f7398142b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 -- 172.21.15.46:0/455120019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7390009b30 con 0x7f739813ff50 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f738ffff640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73980bb1a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f738000d620 tx=0x7f738000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7394ff9640 1 -- 172.21.15.46:0/455120019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000b620 con 0x7f739813ff50 2023-12-16T21:43:49.115 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7394ff9640 1 -- 172.21.15.46:0/455120019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738000b780 con 0x7f739813ff50 2023-12-16T21:43:49.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.107+0000 7f7394ff9640 1 -- 172.21.15.46:0/455120019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000bb10 con 0x7f739813ff50 2023-12-16T21:43:49.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73981423c0 con 0x7f739813ff50 2023-12-16T21:43:49.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73981df530 con 0x7f739813ff50 2023-12-16T21:43:49.116 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7398067de0 con 0x7f739813ff50 2023-12-16T21:43:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f7394ff9640 1 -- 172.21.15.46:0/455120019 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f738000bc70 con 0x7f739813ff50 2023-12-16T21:43:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f7394ff9640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7378066470 0x7f7378068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:43:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f7394ff9640 1 -- 172.21.15.46:0/455120019 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7380084080 con 0x7f739813ff50 2023-12-16T21:43:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f73977fe640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7378066470 0x7f7378068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:43:49.117 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.111+0000 7f73977fe640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7378066470 0x7f7378068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7388000f70 tx=0x7f7388009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:43:49.119 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.115+0000 7f7394ff9640 1 -- 172.21.15.46:0/455120019 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f73800b5290 con 0x7f739813ff50 2023-12-16T21:43:49.273 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.271+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73980bc190 con 0x7f7378066470 2023-12-16T21:43:49.279 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.275+0000 7f7394ff9640 1 -- 172.21.15.46:0/455120019 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+49035 (secure 0 0 0) 0x7f73980bc190 con 0x7f7378066470 2023-12-16T21:43:49.280 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:43:49.281 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:43:49.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7378066470 msgr2=0x7f7378068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:49.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7378066470 0x7f7378068930 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7388000f70 tx=0x7f7388009040 comp rx=0 tx=0).stop 2023-12-16T21:43:49.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 msgr2=0x7f73980bb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:43:49.282 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73980bb1a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f738000d620 tx=0x7f738000dae0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 shutdown_connections 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f7378066470 0x7f7378068930 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f739814a390 0x7f7398142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f739813ff50 0x7f73980bb1a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 --2- 172.21.15.46:0/455120019 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f73980bc800 0x7f7398140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 >> 172.21.15.46:0/455120019 conn(0x7f73980722b0 msgr2=0x7f7398146f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 shutdown_connections 2023-12-16T21:43:49.283 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:43:49.279+0000 7f739dc94640 1 -- 172.21.15.46:0/455120019 wait complete. 2023-12-16T21:43:49.299 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2023-12-16T21:43:48.372788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236952,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737768,"statfs":{"total":773094113280,"available":772851474432,"internally_reserved":0,"allocated":4235264,"data_stored":2351200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10887,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-52,"num_write_kb":-26,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002041"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":177,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-12-16T21:43:40.500436+0000","last_change":"2023-12-16T21:42:31.131965+0000","last_active":"2023-12-16T21:43:40.500436+0000","last_peered":"2023-12-16T21:43:40.500436+0000","last_clean":"2023-12-16T21:43:40.500436+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:40.500436+0000","last_undegraded":"2023-12-16T21:43:40.500436+0000","last_fullsized":"2023-12-16T21:43:40.500436+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-16T21:42:29.196648+0000","last_clean_scrub_stamp":"2023-12-16T21:42:29.196648+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T01:17:15.032172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":974848,"data_stored":736676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82999999999999996}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.0049999999999999}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.879},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.70899999999999996}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.97999999999999998}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":0.751},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":1.0329999999999999}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.78500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":1.143}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.016},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.381}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.427}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.111}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.226},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.3}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":1.179},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.3340000000000001}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.2689999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.90500000000000003}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.86099999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.105},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.0149999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":1.042},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.1930000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":0.97499999999999998}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.0700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":0.498},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.82099999999999995}]},{"osd":1,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.68500000000000005}]},{"osd":2,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.879}]},{"osd":3,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.1339999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":1.208}]},{"osd":6,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.5720000000000001,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":1.052}]},{"osd":7,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":1.1779999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68999999999999995}]},{"osd":1,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.72599999999999998}]},{"osd":2,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.89800000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":0.94099999999999995}]},{"osd":5,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.99099999999999999}]},{"osd":6,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.071}]},{"osd":7,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.119},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":0.96999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.47299999999999998}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.36899999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.44},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":0.501}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.72099999999999997}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":0.65700000000000003},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":0.80200000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.30099999999999999}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.436},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.41699999999999998}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.372},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.35399999999999998}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":0.55500000000000005}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.73899999999999999}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":0.621}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":974848,"data_stored":736676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.441}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.36699999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.46600000000000003}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.629},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":0.72199999999999998}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":0.82799999999999996}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.81100000000000005}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":3.1440000000000001,"5min":3.1440000000000001,"15min":3.1440000000000001},"last":0.75800000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:43:49.302 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-12-16T21:42:29.196648+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=16, tm_hour=21, tm_min=42, tm_sec=29, tm_wday=5, tm_yday=350, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=16, tm_hour=21, tm_min=43, tm_sec=44, tm_wday=5, tm_yday=350, tm_isdst=0) 2023-12-16T21:43:49.303 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-16T21:44:06.790 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:44:06.793 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:44:09.304 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-16T21:44:09.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.430+0000 7f597d4f8640 1 Processor -- start 2023-12-16T21:44:09.437 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f597d4f8640 1 -- start start 2023-12-16T21:44:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f597d4f8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780ae5b0 0x7f59780ae990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f597d4f8640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59780b3230 0x7f59780b3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f597d4f8640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59780b27e0 con 0x7f59780b17a0 2023-12-16T21:44:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f597d4f8640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59780afe40 con 0x7f59780b3230 2023-12-16T21:44:09.438 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f597d4f8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59780b2320 con 0x7f59780ae5b0 2023-12-16T21:44:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780ae5b0 0x7f59780ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f59767fc640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f59780b3230 0x7f59780b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780ae5b0 0x7f59780ae990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.161:3301/0 says I am v2:172.21.15.46:54278/0 (socket says 172.21.15.46:54278) 2023-12-16T21:44:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f5976ffd640 1 -- 172.21.15.46:0/3228999796 learned_addr learned my addr 172.21.15.46:0/3228999796 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:44:09.439 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f5976ffd640 1 -- 172.21.15.46:0/3228999796 >> v2:172.21.15.161:3300/0 conn(0x7f59780b3230 msgr2=0x7f59780b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f5976ffd640 1 --2- 172.21.15.46:0/3228999796 >> v2:172.21.15.161:3300/0 conn(0x7f59780b3230 0x7f59780b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f5976ffd640 1 -- 172.21.15.46:0/3228999796 >> v1:172.21.15.161:6789/0 conn(0x7f59780b17a0 legacy=0x7f59780b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.434+0000 7f5976ffd640 1 -- 172.21.15.46:0/3228999796 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780b3bb0 con 0x7f59780ae5b0 2023-12-16T21:44:09.440 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 --2- 172.21.15.46:0/3228999796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780ae5b0 0x7f59780ae990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098a0 tx=0x7f596c01c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ea0cdadc3bbefc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5975ffb640 1 -- 172.21.15.46:0/3228999796 <== mon.2 v2:172.21.15.161:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0ab660 con 0x7f59780ae5b0 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5975ffb640 1 -- 172.21.15.46:0/3228999796 <== mon.2 v2:172.21.15.161:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c0ab7c0 con 0x7f59780ae5b0 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5975ffb640 1 -- 172.21.15.46:0/3228999796 <== mon.2 v2:172.21.15.161:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0abb50 con 0x7f59780ae5b0 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- 172.21.15.46:0/3228999796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780ae5b0 msgr2=0x7f59780ae990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 --2- 172.21.15.46:0/3228999796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780ae5b0 0x7f59780ae990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098a0 tx=0x7f596c01c460 comp rx=0 tx=0).stop 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- 172.21.15.46:0/3228999796 shutdown_connections 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 --2- 172.21.15.46:0/3228999796 >> v2:172.21.15.161:3300/0 conn(0x7f59780b3230 0x7f59780b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.443 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 --2- 172.21.15.46:0/3228999796 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780ae5b0 0x7f59780ae990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- 172.21.15.46:0/3228999796 >> 172.21.15.46:0/3228999796 conn(0x7f59780b48e0 msgr2=0x7f59780b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:44:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- 172.21.15.46:0/3228999796 shutdown_connections 2023-12-16T21:44:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- 172.21.15.46:0/3228999796 wait complete. 2023-12-16T21:44:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 Processor -- start 2023-12-16T21:44:09.444 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- start start 2023-12-16T21:44:09.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780ae5b0 0x7f5978144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59780b17a0 0x7f59781dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780b3230 0x7f59781dc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5978145240 con 0x7f59780ae5b0 2023-12-16T21:44:09.446 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597814d420 con 0x7f59780b17a0 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5978146a20 con 0x7f59780b3230 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780ae5b0 0x7f5978144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780ae5b0 0x7f5978144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41700/0 (socket says 172.21.15.46:41700) 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 -- 172.21.15.46:0/4059166578 learned_addr learned my addr 172.21.15.46:0/4059166578 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f59767fc640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59780b17a0 0x7f59781dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f59777fe640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780b3230 0x7f59781dc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 -- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780b3230 msgr2=0x7f59781dc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780b3230 0x7f59781dc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.447 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 -- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59780b17a0 msgr2=0x7f59781dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59780b17a0 0x7f59781dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 -- 172.21.15.46:0/4059166578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0094d0 con 0x7f59780ae5b0 2023-12-16T21:44:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f5976ffd640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780ae5b0 0x7f5978144150 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f596c0ad040 tx=0x7f596c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f595f7fe640 1 -- 172.21.15.46:0/4059166578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0b4800 con 0x7f59780ae5b0 2023-12-16T21:44:09.448 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f595f7fe640 1 -- 172.21.15.46:0/4059166578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c0b4980 con 0x7f59780ae5b0 2023-12-16T21:44:09.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597814d140 con 0x7f59780ae5b0 2023-12-16T21:44:09.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.438+0000 7f595f7fe640 1 -- 172.21.15.46:0/4059166578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0b4d30 con 0x7f59780ae5b0 2023-12-16T21:44:09.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.442+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597809cee0 con 0x7f59780ae5b0 2023-12-16T21:44:09.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.442+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59780679e0 con 0x7f59780ae5b0 2023-12-16T21:44:09.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.442+0000 7f595f7fe640 1 -- 172.21.15.46:0/4059166578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f596c0a9070 con 0x7f59780ae5b0 2023-12-16T21:44:09.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.442+0000 7f595f7fe640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5964066470 0x7f5964068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.449 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.442+0000 7f595f7fe640 1 -- 172.21.15.46:0/4059166578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f596c11e310 con 0x7f59780ae5b0 2023-12-16T21:44:09.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.442+0000 7f59767fc640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5964066470 0x7f5964068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.450 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.442+0000 7f59767fc640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5964066470 0x7f5964068930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5978070d50 tx=0x7f5968008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:09.451 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.446+0000 7f595f7fe640 1 -- 172.21.15.46:0/4059166578 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f596c0ebc40 con 0x7f59780ae5b0 2023-12-16T21:44:09.601 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.598+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 --> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f59780ade90 con 0x7f5964066470 2023-12-16T21:44:09.608 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.602+0000 7f595f7fe640 1 -- 172.21.15.46:0/4059166578 <== mgr.4100 v2:172.21.15.161:6832/4278008418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48948 (secure 0 0 0) 0x7f5968011950 con 0x7f5964066470 2023-12-16T21:44:09.609 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:44:09.609 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2023-12-16T21:44:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5964066470 msgr2=0x7f5964068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.610 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5964066470 0x7f5964068930 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5978070d50 tx=0x7f5968008040 comp rx=0 tx=0).stop 2023-12-16T21:44:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780ae5b0 msgr2=0x7f5978144150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780ae5b0 0x7f5978144150 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f596c0ad040 tx=0x7f596c01aca0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 shutdown_connections 2023-12-16T21:44:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f5964066470 0x7f5964068930 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f59780b3230 0x7f59781dc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.611 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f59780b17a0 0x7f59781dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 --2- 172.21.15.46:0/4059166578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59780ae5b0 0x7f5978144150 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 >> 172.21.15.46:0/4059166578 conn(0x7f59780b48e0 msgr2=0x7f5978066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:44:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 shutdown_connections 2023-12-16T21:44:09.612 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.606+0000 7f597d4f8640 1 -- 172.21.15.46:0/4059166578 wait complete. 2023-12-16T21:44:09.626 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-12-16T21:44:08.377995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236936,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737784,"statfs":{"total":773094113280,"available":772851490816,"internally_reserved":0,"allocated":4235264,"data_stored":2351200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002992"},"pg_stats":[{"pgid":"1.0","version":"33'119","reported_seq":188,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-16T21:43:54.817131+0000","last_change":"2023-12-16T21:43:54.817200+0000","last_active":"2023-12-16T21:43:54.817131+0000","last_peered":"2023-12-16T21:43:54.817131+0000","last_clean":"2023-12-16T21:43:54.817131+0000","last_became_active":"2023-12-16T21:42:31.129866+0000","last_became_peered":"2023-12-16T21:42:31.129866+0000","last_unstale":"2023-12-16T21:43:54.817131+0000","last_undegraded":"2023-12-16T21:43:54.817131+0000","last_fullsized":"2023-12-16T21:43:54.817131+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2023-12-16T21:43:54.816926+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2023-12-16T21:43:54.816926+0000","last_clean_scrub_stamp":"2023-12-16T21:43:54.816926+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-18T00:18:08.932316+0000","scrub_duration":0.011207293,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":974848,"data_stored":736676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.0720000000000001}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.022}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.276,"5min":1.276,"15min":1.276},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.1379999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.90400000000000003}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.84499999999999997}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.514,"5min":1.514,"15min":1.514},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":3.2440000000000002,"5min":3.2440000000000002,"15min":3.2440000000000002},"last":0.93899999999999995}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":3.2989999999999999,"5min":3.2989999999999999,"15min":3.2989999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":1.0509999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":1.002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.72699999999999998}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.159}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.78300000000000003}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":1.135}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.871},{"interface":"front","average":{"1min":1.149,"5min":1.149,"15min":1.149},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":1.046}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.226,"5min":1.226,"15min":1.226},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.0820000000000001}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.198,"5min":1.198,"15min":1.198},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.1899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":1.1970000000000001}]},{"osd":1,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.153}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":1.0229999999999999}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":0.995},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":2.198,"5min":2.198,"15min":2.198},"last":1.083}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.855,"5min":2.855,"15min":2.855},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.3860000000000001,"5min":2.3860000000000001,"15min":2.3860000000000001},"last":1.1040000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.085,"5min":1.085,"15min":1.085},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.597,"5min":2.597,"15min":2.597},"last":1.2410000000000001}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":1.129}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.1899999999999999},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.02}]},{"osd":1,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.7050000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.2629999999999999}]},{"osd":2,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.2160000000000002,"5min":2.2160000000000002,"15min":2.2160000000000002},"last":1.2290000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.80800000000000005}]},{"osd":3,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.1279999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.133,"5min":1.133,"15min":1.133},"min":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.8380000000000001,"5min":1.8380000000000001,"15min":1.8380000000000001},"last":0.86699999999999999}]},{"osd":6,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.5720000000000001,"5min":2.5720000000000001,"15min":2.5720000000000001},"last":1.0900000000000001}]},{"osd":7,"last update":"Sat Dec 16 21:43:28 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"max":{"1min":2.133,"5min":2.133,"15min":2.133},"last":1.294},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"min":{"1min":1.004,"5min":1.004,"15min":1.004},"max":{"1min":2.613,"5min":2.613,"15min":2.613},"last":1.05}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.76100000000000001}]},{"osd":1,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.877},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.70899999999999996}]},{"osd":2,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":1.0580000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.81399999999999995}]},{"osd":4,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.9039999999999999,"5min":1.9039999999999999,"15min":1.9039999999999999},"last":1.2729999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.92600000000000005}]},{"osd":6,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.244},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.339}]},{"osd":7,"last update":"Sat Dec 16 21:43:29 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.665,"5min":1.665,"15min":1.665},"last":1.302}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.58799999999999997}]},{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.96899999999999997}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":1.131},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.5609999999999999,"5min":1.5609999999999999,"15min":1.5609999999999999},"last":1.0489999999999999}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.82799999999999996}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.88100000000000001}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.236},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.8089999999999999,"5min":1.8089999999999999,"15min":1.8089999999999999},"last":1.1080000000000001}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.669,"5min":1.669,"15min":1.669},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.0049999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.61299999999999999}]},{"osd":2,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.69599999999999995}]},{"osd":3,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":1.246},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.77500000000000002}]},{"osd":4,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.367,"5min":1.367,"15min":1.367},"last":1.1659999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.026,"5min":1.026,"15min":1.026},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.96899999999999997}]},{"osd":6,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.6839999999999999},"last":1.125}]},{"osd":7,"last update":"Sat Dec 16 21:43:27 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.5469999999999999,"5min":1.5469999999999999,"15min":1.5469999999999999},"last":1.294},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.766,"5min":1.766,"15min":1.766},"last":1.214}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":974848,"data_stored":736676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":1.0900000000000001}]},{"osd":2,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.76400000000000001}]},{"osd":3,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.96599999999999997}]},{"osd":4,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":2.8919999999999999,"5min":2.8919999999999999,"15min":2.8919999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.831,"5min":2.831,"15min":2.831},"last":1.2689999999999999}]},{"osd":5,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6160000000000001,"5min":1.6160000000000001,"15min":1.6160000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.0059999999999998,"5min":3.0059999999999998,"15min":3.0059999999999998},"last":1.01}]},{"osd":6,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":3.0539999999999998,"5min":3.0539999999999998,"15min":3.0539999999999998},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.3120000000000001}]},{"osd":7,"last update":"Sat Dec 16 21:43:30 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.6890000000000001,"5min":2.6890000000000001,"15min":2.6890000000000001},"last":1.232},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":3.1440000000000001,"5min":3.1440000000000001,"15min":3.1440000000000001},"last":1.3440000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-16T21:44:09.629 DEBUG:teuthology.orchestra.run.smithi046:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-16T21:44:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.746+0000 7f9df90bb640 1 Processor -- start 2023-12-16T21:44:09.752 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.746+0000 7f9df90bb640 1 -- start start 2023-12-16T21:44:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df90bb640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f9df41405d0 0x7f9df4152210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df90bb640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df40b4990 con 0x7f9df40bc8a0 2023-12-16T21:44:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df90bb640 1 -- --> v1:172.21.15.161:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df40b5de0 con 0x7f9df414a520 2023-12-16T21:44:09.753 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df90bb640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df40b5b70 con 0x7f9df41405d0 2023-12-16T21:44:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df2d9d640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f9df40bc8a0 0x7f9df4149e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53476/0 (socket says 172.21.15.46:53476) 2023-12-16T21:44:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df2d9d640 1 -- 172.21.15.46:0/3009040066 learned_addr learned my addr 172.21.15.46:0/3009040066 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:44:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 --2- 172.21.15.46:0/3009040066 >> v2:172.21.15.161:3300/0 conn(0x7f9df41405d0 0x7f9df4152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263662140 0 0) 0x7f9df40b4990 con 0x7f9df40bc8a0 2023-12-16T21:44:09.754 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ddc003530 con 0x7f9df40bc8a0 2023-12-16T21:44:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 743237503 0 0) 0x7f9ddc003530 con 0x7f9df40bc8a0 2023-12-16T21:44:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 >> v2:172.21.15.161:3300/0 conn(0x7f9df41405d0 msgr2=0x7f9df4152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 --2- 172.21.15.46:0/3009040066 >> v2:172.21.15.161:3300/0 conn(0x7f9df41405d0 0x7f9df4152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.756 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 >> v1:172.21.15.161:6789/0 conn(0x7f9df414a520 legacy=0x7f9df413fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df40bcc80 con 0x7f9df40bc8a0 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f9de4001680 con 0x7f9df40bc8a0 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 >> v1:172.21.15.46:6789/0 conn(0x7f9df40bc8a0 legacy=0x7f9df4149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc003980 0x7f9ddc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0047d0 0x7f9ddc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc0052b0 0x7f9ddc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40b5b70 con 0x7f9ddc0052b0 2023-12-16T21:44:09.757 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc00c180 con 0x7f9ddc003980 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ddc00c340 con 0x7f9ddc0047d0 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc0052b0 0x7f9ddc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df2d9d640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc003980 0x7f9ddc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 -- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0047d0 msgr2=0x7f9ddc004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df259c640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0047d0 0x7f9ddc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0047d0 0x7f9ddc004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 -- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc003980 msgr2=0x7f9ddc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9ddc003980 0x7f9ddc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.758 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 -- 172.21.15.46:0/3009040066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ddc003530 con 0x7f9ddc0052b0 2023-12-16T21:44:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 -- 172.21.15.46:0/3009040066 reap_dead start 2023-12-16T21:44:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df359e640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc0052b0 0x7f9ddc00b990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009fc0 tx=0x7f9dec01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8f4e2d2a71566f server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0a5c60 con 0x7f9ddc0052b0 2023-12-16T21:44:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec00f040 con 0x7f9ddc0052b0 2023-12-16T21:44:09.759 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.750+0000 7f9df1d9b640 1 -- 172.21.15.46:0/3009040066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0039a0 con 0x7f9ddc0052b0 2023-12-16T21:44:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- 172.21.15.46:0/3009040066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc0052b0 msgr2=0x7f9ddc00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc0052b0 0x7f9ddc00b990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009fc0 tx=0x7f9dec01cc70 comp rx=0 tx=0).stop 2023-12-16T21:44:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- 172.21.15.46:0/3009040066 shutdown_connections 2023-12-16T21:44:09.762 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 --2- 172.21.15.46:0/3009040066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ddc0052b0 0x7f9ddc00b990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- 172.21.15.46:0/3009040066 >> 172.21.15.46:0/3009040066 conn(0x7f9df40722b0 msgr2=0x7f9df40bbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- 172.21.15.46:0/3009040066 shutdown_connections 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- 172.21.15.46:0/3009040066 wait complete. 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 Processor -- start 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- start start 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0052b0 0x7f9df413d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df413c250 0x7f9df41e21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.763 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df413c6d0 0x7f9df41e2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df413d580 con 0x7f9df413c250 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df413e380 con 0x7f9df413c6d0 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df413ba90 con 0x7f9ddc0052b0 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df413c250 0x7f9df41e21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df413c250 0x7f9df41e21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41716/0 (socket says 172.21.15.46:41716) 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 -- 172.21.15.46:0/2649166354 learned_addr learned my addr 172.21.15.46:0/2649166354 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df2d9d640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0052b0 0x7f9df413d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df359e640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df413c6d0 0x7f9df41e2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.764 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 -- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0052b0 msgr2=0x7f9df413d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0052b0 0x7f9df413d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.765 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 -- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df413c6d0 msgr2=0x7f9df41e2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df413c6d0 0x7f9df41e2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 -- 172.21.15.46:0/2649166354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df4141f40 con 0x7f9df413c250 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df2d9d640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0052b0 0x7f9df413d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df259c640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df413c250 0x7f9df41e21d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d540 tx=0x7f9de800da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8005480 con 0x7f9df413c250 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8003620 con 0x7f9df413c250 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df413bd30 con 0x7f9df413c250 2023-12-16T21:44:09.766 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8003990 con 0x7f9df413c250 2023-12-16T21:44:09.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.754+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df40b2730 con 0x7f9df413c250 2023-12-16T21:44:09.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.758+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df4068960 con 0x7f9df413c250 2023-12-16T21:44:09.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.758+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f9de8003af0 con 0x7f9df413c250 2023-12-16T21:44:09.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.758+0000 7f9ddb7fe640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9de0066470 0x7f9de0068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:09.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.758+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9de80839c0 con 0x7f9df413c250 2023-12-16T21:44:09.767 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.758+0000 7f9df2d9d640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9de0066470 0x7f9de0068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.758+0000 7f9df2d9d640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9de0066470 0x7f9de0068930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9de40058c0 tx=0x7f9de4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:09.768 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.762+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f9de80511d0 con 0x7f9df413c250 2023-12-16T21:44:09.926 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.922+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f9df4144020 con 0x7f9df413c250 2023-12-16T21:44:09.931 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.926+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9de80547f0 con 0x7f9df413c250 2023-12-16T21:44:09.932 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9ddb7fe640 1 -- 172.21.15.46:0/2649166354 <== mon.0 v2:172.21.15.46:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9de80511d0 con 0x7f9df413c250 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9de0066470 msgr2=0x7f9de0068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9de0066470 0x7f9de0068930 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9de40058c0 tx=0x7f9de4009040 comp rx=0 tx=0).stop 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df413c250 msgr2=0x7f9df41e21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df413c250 0x7f9df41e21d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9de800d540 tx=0x7f9de800da00 comp rx=0 tx=0).stop 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 shutdown_connections 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f9de0066470 0x7f9de0068930 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f9df413c6d0 0x7f9df41e2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.935 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9df413c250 0x7f9df41e21d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 --2- 172.21.15.46:0/2649166354 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f9ddc0052b0 0x7f9df413d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 >> 172.21.15.46:0/2649166354 conn(0x7f9df40722b0 msgr2=0x7f9df4066360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:44:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 shutdown_connections 2023-12-16T21:44:09.936 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:09.930+0000 7f9df90bb640 1 -- 172.21.15.46:0/2649166354 wait complete. 2023-12-16T21:44:09.951 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-16T21:44:09.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-16T21:44:10.067 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4cb8c640 1 Processor -- start 2023-12-16T21:44:10.068 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4cb8c640 1 -- start start 2023-12-16T21:44:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4cb8c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d480b3fd0 0x7f8d4813e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4cb8c640 1 --2- >> v2:172.21.15.161:3300/0 conn(0x7f8d480b0340 0x7f8d4813e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4cb8c640 1 -- --> v1:172.21.15.46:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480af2e0 con 0x7f8d4806c920 2023-12-16T21:44:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4cb8c640 1 -- --> v2:172.21.15.161:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d480b1f50 con 0x7f8d480b0340 2023-12-16T21:44:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4cb8c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4813eda0 con 0x7f8d480b3fd0 2023-12-16T21:44:10.070 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4659c640 1 --1- >> v1:172.21.15.46:6789/0 conn(0x7f8d4806c920 0x7f8d480afc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.46:6789/0 says I am v1:172.21.15.46:53490/0 (socket says 172.21.15.46:53490) 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.062+0000 7f8d4659c640 1 -- 172.21.15.46:0/2788748778 learned_addr learned my addr 172.21.15.46:0/2788748778 (peer_addr_for_me v1:172.21.15.46:0/0) 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d46d9d640 1 --2- 172.21.15.46:0/2788748778 >> v2:172.21.15.161:3300/0 conn(0x7f8d480b0340 0x7f8d4813e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 <== mon.0 v1:172.21.15.46:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 441930828 0 0) 0x7f8d480af2e0 con 0x7f8d4806c920 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d480b3fd0 0x7f8d4813e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 --> v1:172.21.15.46:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d30003530 con 0x7f8d4806c920 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 <== mon.0 v1:172.21.15.46:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2663682015 0 0) 0x7f8d30003530 con 0x7f8d4806c920 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d480b3fd0 msgr2=0x7f8d4813e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:10.071 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d480b3fd0 0x7f8d4813e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 >> v2:172.21.15.161:3300/0 conn(0x7f8d480b0340 msgr2=0x7f8d4813e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 --2- 172.21.15.46:0/2788748778 >> v2:172.21.15.161:3300/0 conn(0x7f8d480b0340 0x7f8d4813e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 --> v1:172.21.15.46:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4806cd00 con 0x7f8d4806c920 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 <== mon.0 v1:172.21.15.46:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1483241985 0 0) 0x7f8d38001680 con 0x7f8d4806c920 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 >> v1:172.21.15.46:6789/0 conn(0x7f8d4806c920 legacy=0x7f8d480afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d30003980 0x7f8d30003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d30005190 0x7f8d3000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.072 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d480b1f50 con 0x7f8d300047d0 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4813eda0 con 0x7f8d30003980 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d30005680 con 0x7f8d30005190 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4659c640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d30003980 0x7f8d30003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d46d9d640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d30005190 0x7f8d3000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 -- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d30005190 msgr2=0x7f8d3000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d30005190 0x7f8d3000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.073 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 -- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d30003980 msgr2=0x7f8d30003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-16T21:44:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d30003980 0x7f8d30003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.074 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 -- 172.21.15.46:0/2788748778 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30003530 con 0x7f8d300047d0 2023-12-16T21:44:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009ae0 tx=0x7f8d3c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bb26202c9d15d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0a5530 con 0x7f8d300047d0 2023-12-16T21:44:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8d3c0a5690 con 0x7f8d300047d0 2023-12-16T21:44:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4559a640 1 -- 172.21.15.46:0/2788748778 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0a5a60 con 0x7f8d300047d0 2023-12-16T21:44:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/2788748778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 msgr2=0x7f8d30004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:10.078 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4cb8c640 1 --2- 172.21.15.46:0/2788748778 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d30004c50 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c009ae0 tx=0x7f8d3c01ca50 comp rx=0 tx=0).stop 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d46d9d640 1 -- 172.21.15.46:0/2788748778 reap_dead start 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/2788748778 shutdown_connections 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.066+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/2788748778 >> 172.21.15.46:0/2788748778 conn(0x7f8d480b48e0 msgr2=0x7f8d480b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/2788748778 shutdown_connections 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/2788748778 wait complete. 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 Processor -- start 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- start start 2023-12-16T21:44:10.079 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d48141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 --2- >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d4813b3e0 0x7f8d48140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 --2- >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d4813fd20 0x7f8d481401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4813b820 con 0x7f8d300047d0 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- --> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d48141410 con 0x7f8d4813fd20 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- --> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4813ef80 con 0x7f8d4813b3e0 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d48141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d48141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41728/0 (socket says 172.21.15.46:41728) 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 -- 172.21.15.46:0/1261894381 learned_addr learned my addr 172.21.15.46:0/1261894381 (peer_addr_for_me v2:172.21.15.46:0/0) 2023-12-16T21:44:10.080 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d4813b3e0 0x7f8d48140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d46d9d640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d4813fd20 0x7f8d481401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 -- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d4813b3e0 msgr2=0x7f8d48140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d4813b3e0 0x7f8d48140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 -- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d4813fd20 msgr2=0x7f8d481401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d4813fd20 0x7f8d481401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 -- 172.21.15.46:0/1261894381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d48141670 con 0x7f8d300047d0 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d4813b3e0 0x7f8d48140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4659c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d48141de0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e700 tx=0x7f8d3800ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:10.081 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d36ffd640 1 -- 172.21.15.46:0/1261894381 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800ee10 con 0x7f8d300047d0 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d36ffd640 1 -- 172.21.15.46:0/1261894381 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8d3800c8d0 con 0x7f8d300047d0 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4813f220 con 0x7f8d300047d0 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d36ffd640 1 -- 172.21.15.46:0/1261894381 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3800cca0 con 0x7f8d300047d0 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d48139eb0 con 0x7f8d300047d0 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.070+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d480679e0 con 0x7f8d300047d0 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.074+0000 7f8d36ffd640 1 -- 172.21.15.46:0/1261894381 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) v1 ==== 85854+0+0 (secure 0 0 0) 0x7f8d3801d460 con 0x7f8d300047d0 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.074+0000 7f8d36ffd640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8d280664a0 0x7f8d28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-16T21:44:10.082 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.074+0000 7f8d36ffd640 1 -- 172.21.15.46:0/1261894381 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d38083b50 con 0x7f8d300047d0 2023-12-16T21:44:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.074+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8d280664a0 0x7f8d28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-16T21:44:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.074+0000 7f8d45d9b640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8d280664a0 0x7f8d28068960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0af000 tx=0x7f8d3c0046d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-16T21:44:10.083 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.078+0000 7f8d36ffd640 1 -- 172.21.15.46:0/1261894381 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172666 (secure 0 0 0) 0x7f8d38054cd0 con 0x7f8d300047d0 2023-12-16T21:44:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.266+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d480688f0 con 0x7f8d300047d0 2023-12-16T21:44:10.271 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.266+0000 7f8d36ffd640 1 -- 172.21.15.46:0/1261894381 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2120 (secure 0 0 0) 0x7f8d38001570 con 0x7f8d300047d0 2023-12-16T21:44:10.272 INFO:teuthology.orchestra.run.smithi046.stdout: 2023-12-16T21:44:10.272 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4607,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.161:6835/3332084711","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.161:6834","nonce":3332084711},{"type":"v1","addr":"172.21.15.161:6835","nonce":3332084711}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5181,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6833/2602182348","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2602182348},{"type":"v1","addr":"172.21.15.46:6833","nonce":2602182348}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[]} 2023-12-16T21:44:10.272 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 15 2023-12-16T21:44:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8d280664a0 msgr2=0x7f8d28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8d280664a0 0x7f8d28068960 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0af000 tx=0x7f8d3c0046d0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 msgr2=0x7f8d48141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-16T21:44:10.274 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d48141de0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8d3800e700 tx=0x7f8d3800ebc0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 shutdown_connections 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:6832/4278008418,v1:172.21.15.161:6833/4278008418] conn(0x7f8d280664a0 0x7f8d28068960 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3300/0,v1:172.21.15.161:6789/0] conn(0x7f8d4813fd20 0x7f8d481401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.161:3301/0,v1:172.21.15.161:6790/0] conn(0x7f8d4813b3e0 0x7f8d48140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 --2- 172.21.15.46:0/1261894381 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8d300047d0 0x7f8d48141de0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 >> 172.21.15.46:0/1261894381 conn(0x7f8d480b48e0 msgr2=0x7f8d4813cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 shutdown_connections 2023-12-16T21:44:10.275 INFO:teuthology.orchestra.run.smithi046.stderr:2023-12-16T21:44:10.270+0000 7f8d4cb8c640 1 -- 172.21.15.46:0/1261894381 wait complete. 2023-12-16T21:44:10.292 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-16T21:44:10.293 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-16T21:44:10.293 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-16T21:44:10.293 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:10.307 INFO:tasks.ceph.mds.a:Stopped 2023-12-16T21:44:10.307 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-16T21:44:10.307 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:10.331 INFO:tasks.ceph.mds.b:Stopped 2023-12-16T21:44:10.331 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-16T21:44:10.332 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-16T21:44:10.332 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:10.431 INFO:tasks.ceph.osd.0:Stopped 2023-12-16T21:44:10.431 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-16T21:44:10.431 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:16.534 INFO:tasks.ceph.osd.1:Stopped 2023-12-16T21:44:16.534 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-16T21:44:16.534 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:16.592 INFO:tasks.ceph.osd.2:Stopped 2023-12-16T21:44:16.593 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-16T21:44:16.593 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:16.857 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-12-16T21:44:22.695 INFO:tasks.ceph.osd.3:Stopped 2023-12-16T21:44:22.695 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-16T21:44:22.695 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:22.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-12-16T21:44:22.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-12-16T21:44:22.770 INFO:tasks.ceph.osd.4:Stopped 2023-12-16T21:44:22.771 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-16T21:44:22.771 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:22.860 INFO:tasks.ceph.osd.5:Stopped 2023-12-16T21:44:22.860 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-16T21:44:22.860 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:22.947 INFO:tasks.ceph.osd.6:Stopped 2023-12-16T21:44:22.947 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-16T21:44:22.947 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:23.019 INFO:tasks.ceph.osd.7:Stopped 2023-12-16T21:44:23.019 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-16T21:44:23.020 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-16T21:44:23.020 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:23.048 INFO:tasks.ceph.mgr.y:Stopped 2023-12-16T21:44:23.048 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-16T21:44:23.048 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:23.077 INFO:tasks.ceph.mgr.x:Stopped 2023-12-16T21:44:23.078 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-16T21:44:23.078 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-16T21:44:23.078 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:23.107 INFO:tasks.ceph.mon.a:Stopped 2023-12-16T21:44:23.107 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-16T21:44:23.107 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:23.134 INFO:tasks.ceph.mon.b:Stopped 2023-12-16T21:44:23.134 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-16T21:44:23.134 INFO:teuthology.orchestra.run:waiting for 300 2023-12-16T21:44:23.193 INFO:tasks.ceph.mon.c:Stopped 2023-12-16T21:44:23.193 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-16T21:44:23.194 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-16T21:44:23.211 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:44:23.211 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-16T21:44:23.411 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:44:23.412 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-16T21:44:23.669 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:44:23.669 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-16T21:44:23.811 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:44:23.812 DEBUG:teuthology.orchestra.run.smithi046:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-16T21:44:23.936 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:44:23.936 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-16T21:44:24.230 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:44:24.230 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-16T21:44:24.403 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:44:24.403 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-16T21:44:24.551 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:44:24.552 DEBUG:teuthology.orchestra.run.smithi161:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-16T21:44:24.710 INFO:tasks.ceph:Archiving mon data... 2023-12-16T21:44:24.712 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/data/mon.a.tgz 2023-12-16T21:44:24.712 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2023-12-16T21:44:24.718 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.WXR4aUpIQV 2023-12-16T21:44:24.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WXR4aUpIQV 2023-12-16T21:44:24.834 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.WXR4aUpIQV 2023-12-16T21:44:24.897 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.WXR4aUpIQV is 273KB 2023-12-16T21:44:24.958 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.WXR4aUpIQV 2023-12-16T21:44:24.964 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/data/mon.b.tgz 2023-12-16T21:44:24.965 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-12-16T21:44:24.970 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.hQjtoNlX4b 2023-12-16T21:44:24.971 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.hQjtoNlX4b 2023-12-16T21:44:25.102 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.hQjtoNlX4b 2023-12-16T21:44:25.167 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.hQjtoNlX4b is 362KB 2023-12-16T21:44:25.227 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.hQjtoNlX4b 2023-12-16T21:44:25.233 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/data/mon.c.tgz 2023-12-16T21:44:25.234 DEBUG:teuthology.orchestra.run.smithi161:> mktemp 2023-12-16T21:44:25.278 INFO:teuthology.orchestra.run.smithi161.stdout:/tmp/tmp.x5LlqqYVea 2023-12-16T21:44:25.278 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.x5LlqqYVea 2023-12-16T21:44:25.408 DEBUG:teuthology.orchestra.run.smithi161:> sudo chmod 0666 /tmp/tmp.x5LlqqYVea 2023-12-16T21:44:25.470 DEBUG:teuthology.orchestra.remote:smithi161:/tmp/tmp.x5LlqqYVea is 363KB 2023-12-16T21:44:25.532 DEBUG:teuthology.orchestra.run.smithi161:> rm -fr /tmp/tmp.x5LlqqYVea 2023-12-16T21:44:25.537 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-16T21:44:25.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-16T21:44:25.540 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-16T21:44:25.601 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2023-12-16T21:44:25.643 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2023-12-16T21:44:25.643 INFO:tasks.ceph:Archiving crash dumps... 2023-12-16T21:44:25.646 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/remote/smithi046/crash 2023-12-16T21:44:25.647 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-16T21:44:25.661 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/remote/smithi161/crash 2023-12-16T21:44:25.662 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-16T21:44:25.698 INFO:tasks.ceph:Shutting down logrotate 2023-12-16T21:44:25.698 DEBUG:teuthology.orchestra.run.smithi046:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:44:25.706 DEBUG:teuthology.orchestra.run.smithi161:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-16T21:44:25.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-16T21:44:25.798 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-16T21:44:25.837 INFO:tasks.ceph:Compressing logs... 2023-12-16T21:44:25.837 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-16T21:44:25.842 DEBUG:teuthology.orchestra.run.smithi161:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-12-16T21:44:25.859 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44476.log 2023-12-16T21:44:25.860 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47723.log 2023-12-16T21:44:25.860 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.0.44476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2023-12-16T21:44:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45633.log 2023-12-16T21:44:25.861 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.0.44476.log.gz 88.4% 2023-12-16T21:44:25.861 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45503.log: -- replaced with /var/log/ceph/ceph-client.admin.47723.log.gz 2023-12-16T21:44:25.862 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2023-12-16T21:44:25.863 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2023-12-16T21:44:25.863 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45425.log 2023-12-16T21:44:25.863 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46294.log 2023-12-16T21:44:25.863 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45633.log.gz 2023-12-16T21:44:25.863 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45425.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz --verbose 2023-12-16T21:44:25.863 INFO:teuthology.orchestra.run.smithi046.stderr: -- /var/log/ceph/ceph-client.admin.43255.log 2023-12-16T21:44:25.864 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45425.log.gz 2023-12-16T21:44:25.865 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46294.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-12-16T21:44:25.865 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46294.log.gz 2023-12-16T21:44:25.865 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47385.log 2023-12-16T21:44:25.865 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43255.log.gz 2023-12-16T21:44:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44087.log 2023-12-16T21:44:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43983.log 2023-12-16T21:44:25.866 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47385.log.gz 2023-12-16T21:44:25.866 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46102.log 2023-12-16T21:44:25.866 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44087.log.gz 2023-12-16T21:44:25.869 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43069.log 2023-12-16T21:44:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43983.log: /var/log/ceph/ceph-client.admin.46102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43508.log 2023-12-16T21:44:25.870 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43983.log.gz 2023-12-16T21:44:25.870 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46102.log.gz 2023-12-16T21:44:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43715.log 2023-12-16T21:44:25.870 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43069.log.gz 2023-12-16T21:44:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42963.log 2023-12-16T21:44:25.870 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43715.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43508.log.gz 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43715.log.gz 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45236.log 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42963.log.gz 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46553.log 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43726.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45236.log.gz 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr: 92.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-12-16T21:44:25.871 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47094.log 2023-12-16T21:44:25.872 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2023-12-16T21:44:25.872 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43121.log 2023-12-16T21:44:25.878 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47094.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46553.log.gz 2023-12-16T21:44:25.878 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43499.log 2023-12-16T21:44:25.878 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47094.log.gz/var/log/ceph/ceph-client.admin.43121.log: 2023-12-16T21:44:25.878 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45262.log 2023-12-16T21:44:25.878 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43121.log.gz 2023-12-16T21:44:25.878 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44968.log 2023-12-16T21:44:25.878 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45262.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43499.log.gz -5 2023-12-16T21:44:25.879 INFO:teuthology.orchestra.run.smithi046.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43517.log 2023-12-16T21:44:25.879 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44968.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45262.log.gz 2023-12-16T21:44:25.879 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38427.log 2023-12-16T21:44:25.879 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44968.log.gz 2023-12-16T21:44:25.879 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43411.log 2023-12-16T21:44:25.879 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46074.log 2023-12-16T21:44:25.879 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43517.log.gz 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6%/var/log/ceph/ceph-client.admin.43411.log: -- replaced with /var/log/ceph/ceph-client.admin.38427.log.gz 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43411.log.gz 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45864.log 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43281.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46074.log.gz 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44890.log 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2023-12-16T21:44:25.880 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43697.log 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45864.log.gz 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44165.log 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43697.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.44890.log.gz 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47697.log 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44165.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43697.log.gz 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43307.log 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.44165.log.gz 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45994.log 2023-12-16T21:44:25.881 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44295.log 2023-12-16T21:44:25.882 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47697.log.gz 2023-12-16T21:44:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45994.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43307.log.gz 2023-12-16T21:44:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46749.log 2023-12-16T21:44:25.883 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45994.log.gz 2023-12-16T21:44:25.883 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43385.log 2023-12-16T21:44:25.896 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43717.log 2023-12-16T21:44:25.896 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43385.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46749.log.gz 2023-12-16T21:44:25.896 INFO:teuthology.orchestra.run.smithi046.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44295.log.gz 2023-12-16T21:44:25.896 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44113.log 2023-12-16T21:44:25.896 INFO:teuthology.orchestra.run.smithi046.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43385.log.gz 2023-12-16T21:44:25.896 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43177.log 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43717.log.gz 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47016.log 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42917.log 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44113.log.gz 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43177.log.gz 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47879.log 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47016.log.gz 2023-12-16T21:44:25.897 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42917.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47879.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42917.log.gz 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44217.log 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47879.log.gz 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44838.log 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45451.log 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44838.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44217.log.gz 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45786.log 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44838.log.gz 2023-12-16T21:44:25.898 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47775.log 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45451.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45786.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45451.log.gz 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45916.log 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45786.log.gz 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47775.log.gz 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43437.log 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47515.log 2023-12-16T21:44:25.899 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45916.log.gz 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43437.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33383.log 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47829.log 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43437.log.gz 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47515.log.gz 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.tmp-client.admin.33383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47333.log 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33383.log.gz 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43957.log 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47829.log.gz 2023-12-16T21:44:25.900 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45131.log 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43716.log 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43957.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47333.log.gz 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44035.log 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43957.log.gz 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45131.log.gz 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45371.log 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45020.log 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43716.log.gz 2023-12-16T21:44:25.901 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44035.log.gz 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43515.log 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45020.log: /var/log/ceph/ceph-client.admin.43149.log 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45371.log.gz 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43515.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45020.log.gz 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45105.log 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43515.log.gz 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46723.log 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45183.log 2023-12-16T21:44:25.902 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43149.log.gz 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46398.log 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45105.log.gz 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46723.log: /var/log/ceph/ceph-client.admin.45183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47593.log 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46723.log.gz 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45183.log.gz 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46858.log 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7%/var/log/ceph/ceph-client.admin.47593.log: -- replaced with /var/log/ceph/ceph-client.admin.46398.log.gz 2023-12-16T21:44:25.903 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47593.log.gz 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45685.log 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47042.log 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45740.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46858.log.gz 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45685.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.47042.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44734.log 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45685.log.gz 2023-12-16T21:44:25.904 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.3.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47042.log.gz 2023-12-16T21:44:25.905 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43516.log 2023-12-16T21:44:25.905 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47853.log 2023-12-16T21:44:25.905 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44734.log: /var/log/ceph/ceph-client.admin.43516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43333.log 2023-12-16T21:44:25.918 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44734.log.gz 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43725.log 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43516.log.gz 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47853.log.gz 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45314.log 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43333.log.gz 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47068.log 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43725.log: /var/log/ceph/ceph-client.admin.45314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46162.log 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43725.log.gz 2023-12-16T21:44:25.919 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43463.log 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45314.log.gz 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47068.log.gz 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46162.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46162.log.gz 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45209.log 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45046.log 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr:gzip/var/log/ceph/ceph-client.admin.45209.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46424.log 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43463.log.gz 2023-12-16T21:44:25.920 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45046.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45209.log.gz 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46424.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47411.log 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45046.log.gz 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46424.log.gz 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46136.log 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46320.log 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47411.log.gz 2023-12-16T21:44:25.921 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44191.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46136.log.gz 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46609.log 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43707.log 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7%/var/log/ceph/ceph-client.0.46609.log: -- replaced with /var/log/ceph/ceph-client.admin.46320.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44812.log 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43707.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43707.log.gz 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44916.log 2023-12-16T21:44:25.922 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46020.log 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44916.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44812.log.gz 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44916.log.gz 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44243.log 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44061.log 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.46609.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46020.log.gz 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44760.log 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44243.log.gz 2023-12-16T21:44:25.923 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43905.log 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44009.log 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44061.log.gz 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43905.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44760.log.gz 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46476.log 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43905.log.gz 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44139.log 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44630.log 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5%/var/log/ceph/ceph-client.admin.46476.log: -- replaced with /var/log/ceph/ceph-client.admin.44009.log.gz 2023-12-16T21:44:25.924 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44139.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46476.log.gz 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45968.log 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44139.log.gz 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46268.log 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45711.log 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44630.log.gz 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46268.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45968.log.gz 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47281.log 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46268.log.gz 2023-12-16T21:44:25.925 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45711.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47281.log: /var/log/ceph/ceph.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45711.log.gz 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47281.log.gz 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47645.log 85.5% 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46962.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2023-12-16T21:44:25.926 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46372.log 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44656.log 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47645.log: /var/log/ceph/ceph-client.admin.46372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45607.log 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47645.log.gz 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7%/var/log/ceph/ceph-client.admin.44656.log: -- replaced with /var/log/ceph/ceph-client.admin.46372.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43514.log 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45607.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44656.log.gz 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45607.log.gz 2023-12-16T21:44:25.927 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44708.log 2023-12-16T21:44:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44526.log 2023-12-16T21:44:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43514.log: /var/log/ceph/ceph-client.admin.44708.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43514.log.gz 2023-12-16T21:44:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46775.log 2023-12-16T21:44:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44526.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44708.log.gz 2023-12-16T21:44:25.939 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44526.log.gz 2023-12-16T21:44:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46697.log 2023-12-16T21:44:25.939 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44269.log 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46775.log: /var/log/ceph/ceph-client.admin.46697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45072.log 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46775.log.gz 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9%/var/log/ceph/ceph-client.admin.44269.log: -- replaced with /var/log/ceph/ceph-client.admin.46697.log.gz 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46216.log 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9%/var/log/ceph/ceph-client.admin.45072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45399.log 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44269.log.gz 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46216.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45072.log.gz 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2023-12-16T21:44:25.940 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46216.log.gz 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42989.log 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45399.log: /var/log/ceph/ceph-client.admin.47217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46346.log 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42989.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45399.log.gz 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47489.log 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42989.log.gz 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44994.log 2023-12-16T21:44:25.941 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46346.log: /var/log/ceph/ceph-client.admin.47489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46936.log 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44994.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46346.log.gz 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47489.log.gz 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45555.log 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44994.log.gz 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45812.log 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46936.log: /var/log/ceph/ceph-client.admin.45555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45529.log 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45812.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46936.log.gz 2023-12-16T21:44:25.942 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45555.log.gz 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43931.log 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45812.log.gz 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47437.log 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8%/var/log/ceph/ceph-client.admin.43931.log: -- replaced with /var/log/ceph/ceph-client.admin.45529.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47128.log 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43931.log.gz 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47437.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47437.log.gz 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43504.log 2023-12-16T21:44:25.943 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47128.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7%/var/log/ceph/ceph-client.admin.43504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47541.log 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47128.log.gz 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43504.log.gz 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph.audit.log: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46450.log 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45890.log 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46450.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47541.log.gz 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45659.log 2023-12-16T21:44:25.944 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46450.log.gz 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45890.log: /var/log/ceph/ceph-client.admin.45659.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45890.log.gz 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45659.log.gz 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43203.log 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47463.log 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47801.log 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47463.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43203.log.gz 2023-12-16T21:44:25.945 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47463.log.gz 2023-12-16T21:44:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38191.log 2023-12-16T21:44:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44864.log 2023-12-16T21:44:25.946 INFO:teuthology.orchestra.run.smithi046.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47801.log.gz 2023-12-16T21:44:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.38191.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.38191.log.gz 2023-12-16T21:44:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45477.log 2023-12-16T21:44:25.946 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47243.log 2023-12-16T21:44:25.953 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44864.log: /var/log/ceph/ceph-client.admin.45477.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44864.log.gz 2023-12-16T21:44:25.953 INFO:teuthology.orchestra.run.smithi046.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45477.log.gz 2023-12-16T21:44:25.953 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45942.log 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47191.log 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47243.log: /var/log/ceph/ceph-client.admin.45942.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47243.log.gz 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45942.log.gz 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46503.log 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44942.log 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47191.log: /var/log/ceph/ceph-client.admin.46503.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47191.log.gz 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.46503.log.gz 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46667.log 2023-12-16T21:44:25.954 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44942.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44942.log.gz 2023-12-16T21:44:25.955 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47165.log 2023-12-16T21:44:25.955 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46801.log 2023-12-16T21:44:25.955 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46667.log: /var/log/ceph/ceph-client.admin.47165.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46667.log.gz 2023-12-16T21:44:25.955 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47165.log.gz 2023-12-16T21:44:25.955 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44350.log 2023-12-16T21:44:25.955 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42830.log 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46801.log: /var/log/ceph/ceph-client.admin.44350.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46801.log.gz 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44350.log.gz 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46910.log 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44552.log 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.42830.log: /var/log/ceph/ceph-client.admin.46910.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42830.log.gz 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46910.log.gz 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43359.log 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47567.log 2023-12-16T21:44:25.956 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43359.log: /var/log/ceph/ceph-client.admin.44552.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43359.log.gz 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44552.log.gz 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45157.log 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44786.log 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47567.log: /var/log/ceph/ceph-client.admin.45157.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47567.log.gz 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45157.log.gz 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46242.log 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46188.log 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44786.log: /var/log/ceph/ceph-client.admin.46242.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44786.log.gz 2023-12-16T21:44:25.957 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46242.log.gz 2023-12-16T21:44:25.958 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45581.log 2023-12-16T21:44:25.958 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2023-12-16T21:44:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46188.log: /var/log/ceph/ceph-client.admin.45581.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46188.log.gz 2023-12-16T21:44:25.963 INFO:teuthology.orchestra.run.smithi046.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45581.log.gz 2023-12-16T21:44:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43017.log 2023-12-16T21:44:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46832.log 2023-12-16T21:44:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.44682.log: /var/log/ceph/ceph-client.admin.43017.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2023-12-16T21:44:25.963 INFO:teuthology.orchestra.run.smithi046.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43017.log.gz 2023-12-16T21:44:25.963 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45288.log 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46046.log 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46832.log: /var/log/ceph/ceph-client.admin.45288.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46832.log.gz 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45288.log.gz 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47619.log 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43229.log 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46046.log: /var/log/ceph/ceph-client.admin.47619.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46046.log.gz 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47619.log.gz 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47359.log 2023-12-16T21:44:25.964 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47307.log 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43229.log: /var/log/ceph/ceph-client.admin.47359.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43229.log.gz 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47359.log.gz 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42887.log 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46988.log 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47307.log: /var/log/ceph/ceph-client.admin.42887.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47307.log.gz 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42887.log.gz 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45340.log 2023-12-16T21:44:25.965 INFO:teuthology.orchestra.run.smithi046.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-12-16T21:44:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46988.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46988.log.gz 2023-12-16T21:44:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47671.log 2023-12-16T21:44:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2023-12-16T21:44:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.47671.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45340.log.gz 2023-12-16T21:44:25.966 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43095.log 2023-12-16T21:44:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43043.log 2023-12-16T21:44:25.972 INFO:teuthology.orchestra.run.smithi046.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47671.log.gz 2023-12-16T21:44:25.972 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.46884.log: /var/log/ceph/ceph-client.admin.43095.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43095.log.gz 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44578.log 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38347.log 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43043.log.gz/var/log/ceph/ceph-client.admin.44578.log: 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45838.log 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43714.log 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr: 89.1%/var/log/ceph/ceph-client.admin.38347.log: -- replaced with /var/log/ceph/ceph-client.admin.44578.log.gz 2023-12-16T21:44:25.973 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.45838.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38347.log.gz 2023-12-16T21:44:25.974 INFO:teuthology.orchestra.run.smithi046.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45838.log.gz 2023-12-16T21:44:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47749.log 2023-12-16T21:44:25.974 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ceph-client.admin.43714.log: /var/log/ceph/ceph-client.admin.47749.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43714.log.gz 2023-12-16T21:44:25.974 INFO:teuthology.orchestra.run.smithi046.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47749.log.gz 2023-12-16T21:44:25.977 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38436.log 2023-12-16T21:44:25.977 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38515.log 2023-12-16T21:44:25.977 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.38436.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-12-16T21:44:25.977 INFO:teuthology.orchestra.run.smithi161.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38436.log.gz 2023-12-16T21:44:25.977 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.38515.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-12-16T21:44:25.977 INFO:teuthology.orchestra.run.smithi161.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.38515.log.gz 2023-12-16T21:44:25.977 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38364.log 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.38364.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38364.log.gz 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38539.log 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.38412.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.38539.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38539.log.gz 2023-12-16T21:44:25.978 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38460.log 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.38460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38484.log 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr: 87.1%/var/log/ceph/ceph.audit.log: -- replaced with /var/log/ceph/ceph-client.admin.38460.log.gz 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38388.log 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.38484.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38484.log.gz 2023-12-16T21:44:25.979 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-client.admin.38388.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38388.log.gz 2023-12-16T21:44:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-12-16T21:44:25.980 INFO:teuthology.orchestra.run.smithi161.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-12-16T21:44:25.980 INFO:teuthology.orchestra.run.smithi161.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-12-16T21:44:25.980 INFO:teuthology.orchestra.run.smithi046.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-12-16T21:44:25.987 INFO:teuthology.orchestra.run.smithi161.stderr:/var/log/ceph/ceph-mds.b.log: /var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-osd.4.log: 90.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-12-16T21:44:25.988 INFO:teuthology.orchestra.run.smithi046.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-12-16T21:44:26.010 INFO:teuthology.orchestra.run.smithi046.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-12-16T21:44:26.012 INFO:teuthology.orchestra.run.smithi161.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-12-16T21:44:26.023 INFO:teuthology.orchestra.run.smithi046.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-12-16T21:44:26.057 INFO:teuthology.orchestra.run.smithi161.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-12-16T21:44:26.058 INFO:teuthology.orchestra.run.smithi161.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-12-16T21:44:26.060 INFO:teuthology.orchestra.run.smithi161.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-12-16T21:44:26.067 INFO:teuthology.orchestra.run.smithi046.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-12-16T21:44:26.069 INFO:teuthology.orchestra.run.smithi046.stderr: 2023-12-16T21:44:26.069 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m0.225s 2023-12-16T21:44:26.069 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m0.828s 2023-12-16T21:44:26.069 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m0.113s 2023-12-16T21:44:26.079 INFO:teuthology.orchestra.run.smithi161.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-12-16T21:44:26.097 INFO:teuthology.orchestra.run.smithi161.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-12-16T21:44:26.122 INFO:teuthology.orchestra.run.smithi161.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-12-16T21:44:26.124 INFO:teuthology.orchestra.run.smithi161.stderr: 2023-12-16T21:44:26.124 INFO:teuthology.orchestra.run.smithi161.stderr:real 0m0.236s 2023-12-16T21:44:26.124 INFO:teuthology.orchestra.run.smithi161.stderr:user 0m0.717s 2023-12-16T21:44:26.124 INFO:teuthology.orchestra.run.smithi161.stderr:sys 0m0.045s 2023-12-16T21:44:26.125 INFO:tasks.ceph:Archiving logs... 2023-12-16T21:44:26.125 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/remote/smithi046/log 2023-12-16T21:44:26.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-16T21:44:26.481 DEBUG:teuthology.misc:Transferring archived files from smithi161:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/remote/smithi161/log 2023-12-16T21:44:26.482 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-16T21:44:26.572 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-16T21:44:26.586 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-16T21:44:26.586 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-16T21:44:26.589 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-16T21:44:26.667 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-16T21:44:26.680 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-16T21:44:26.680 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-16T21:44:26.683 DEBUG:teuthology.orchestra.run.smithi161:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-16T21:44:26.695 INFO:teuthology.orchestra.run.smithi046.stdout: remote refid st t when poll reach delay offset jitter 2023-12-16T21:44:26.695 INFO:teuthology.orchestra.run.smithi046.stdout:============================================================================== 2023-12-16T21:44:26.696 INFO:teuthology.orchestra.run.smithi046.stdout:+hv01.front.sepi 44.4.53.2 2 u 60 64 37 0.076 -0.337 0.386 2023-12-16T21:44:26.696 INFO:teuthology.orchestra.run.smithi046.stdout:+hv02.front.sepi 63.231.80.2 3 u 57 64 37 0.092 -0.241 0.219 2023-12-16T21:44:26.696 INFO:teuthology.orchestra.run.smithi046.stdout:*hv03.front.sepi 74.6.168.72 3 u 61 64 37 0.114 -0.292 0.202 2023-12-16T21:44:26.696 INFO:teuthology.orchestra.run.smithi046.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:44:26.698 INFO:teuthology.orchestra.run.smithi161.stdout: remote refid st t when poll reach delay offset jitter 2023-12-16T21:44:26.698 INFO:teuthology.orchestra.run.smithi161.stdout:============================================================================== 2023-12-16T21:44:26.698 INFO:teuthology.orchestra.run.smithi161.stdout:*hv01.front.sepi 44.4.53.2 2 u 62 64 37 0.097 -1.124 0.871 2023-12-16T21:44:26.699 INFO:teuthology.orchestra.run.smithi161.stdout:+hv02.front.sepi 63.231.80.2 3 u 62 64 37 0.088 -1.176 0.855 2023-12-16T21:44:26.699 INFO:teuthology.orchestra.run.smithi161.stdout:+hv03.front.sepi 74.6.168.72 3 u 60 64 37 0.112 -1.350 0.862 2023-12-16T21:44:26.699 INFO:teuthology.orchestra.run.smithi161.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-16T21:44:26.699 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-16T21:44:26.708 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-16T21:44:26.709 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-16T21:44:26.717 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-16T21:44:26.727 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-16T21:44:26.796 INFO:teuthology.task.internal:Duration was 806.890195 seconds 2023-12-16T21:44:26.797 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-16T21:44:26.806 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-16T21:44:26.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-16T21:44:26.809 DEBUG:teuthology.orchestra.run.smithi161:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-16T21:44:26.847 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-16T21:44:26.847 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2023-12-16T21:44:26.848 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-16T21:44:26.898 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi161.front.sepia.ceph.com 2023-12-16T21:44:26.898 DEBUG:teuthology.orchestra.run.smithi161:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-16T21:44:26.907 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-16T21:44:26.908 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-16T21:44:26.942 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-16T21:44:26.970 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-16T21:44:26.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-16T21:44:27.006 DEBUG:teuthology.orchestra.run.smithi161:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-16T21:44:27.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-16T21:44:27.076 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-16T21:44:27.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-16T21:44:27.091 DEBUG:teuthology.orchestra.run.smithi161:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-16T21:44:27.102 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-16T21:44:27.115 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-16T21:44:27.134 DEBUG:teuthology.orchestra.run.smithi161:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-16T21:44:27.143 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2023-12-16T21:44:27.155 INFO:teuthology.orchestra.run.smithi161.stdout:kernel.core_pattern = core 2023-12-16T21:44:27.181 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-16T21:44:27.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:44:27.207 DEBUG:teuthology.orchestra.run.smithi161:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-16T21:44:27.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-16T21:44:27.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-16T21:44:27.240 INFO:teuthology.task.internal:Transferring archived files... 2023-12-16T21:44:27.241 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/remote/smithi046 2023-12-16T21:44:27.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-16T21:44:27.268 DEBUG:teuthology.misc:Transferring archived files from smithi161:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866/remote/smithi161 2023-12-16T21:44:27.269 DEBUG:teuthology.orchestra.run.smithi161:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-16T21:44:27.292 INFO:teuthology.task.internal:Removing archive directory... 2023-12-16T21:44:27.293 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-16T21:44:27.310 DEBUG:teuthology.orchestra.run.smithi161:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-16T21:44:27.334 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-16T21:44:27.344 INFO:teuthology.task.internal:Not uploading archives. 2023-12-16T21:44:27.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-16T21:44:27.355 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-16T21:44:27.356 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-16T21:44:27.359 DEBUG:teuthology.orchestra.run.smithi161:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-16T21:44:27.363 INFO:teuthology.orchestra.run.smithi046.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 16 21:44 /home/ubuntu/cephtest 2023-12-16T21:44:27.382 INFO:teuthology.orchestra.run.smithi161.stdout: 655472 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 16 21:44 /home/ubuntu/cephtest 2023-12-16T21:44:27.384 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-16T21:44:27.396 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-16T21:44:27.417 INFO:teuthology.nuke:Checking targets against current locks 2023-12-16T21:44:27.445 DEBUG:teuthology.nuke:shortname: smithi046 2023-12-16T21:44:27.445 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-16T21:44:27.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 21:21:49.522323', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwIPeHm8uOK67B7OzSC8aqMeMUOaYZkPomxzmiewZk1jLW5r1rcwjDi/6KEnxjGGz0NDRq/V4VbNnS8qtp6GvU='} 2023-12-16T21:44:27.475 DEBUG:teuthology.nuke:shortname: smithi161 2023-12-16T21:44:27.475 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-16T21:44:27.499 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi161.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-12-16_15:45:14-fs-wip-yuri8-testing-2023-12-15-0911-distro-default-smithi/7494866', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-16 21:21:49.524149', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGXadOoxwcYUpBOvDOHwRreOuQo+PCOSwcoQjSsc62RmCVySnCVdR6sXLE9o3/vznW6S63J92o/drQ/FMSVczVk='} 2023-12-16T21:44:27.528 INFO:teuthology.orchestra.console.smithi046:Power off 2023-12-16T21:44:27.529 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-16T21:44:27.555 DEBUG:teuthology.orchestra.console.smithi046:power off output: Chassis Power Control: Down/Off 2023-12-16T21:44:27.555 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T21:44:27.576 INFO:teuthology.orchestra.console.smithi161:Power off 2023-12-16T21:44:27.576 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-16T21:44:27.588 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-12-16T21:44:27.598 DEBUG:teuthology.orchestra.console.smithi161:power off output: Chassis Power Control: Down/Off 2023-12-16T21:44:27.598 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T21:44:27.624 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2023-12-16T21:44:27.765 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-12-16T21:44:31.589 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T21:44:31.625 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T21:44:31.703 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is on 2023-12-16T21:44:31.736 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is on 2023-12-16T21:44:35.705 DEBUG:teuthology.orchestra.console.smithi046:pexpect command: ipmitool -H smithi046.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T21:44:35.737 DEBUG:teuthology.orchestra.console.smithi161:pexpect command: ipmitool -H smithi161.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-16T21:44:35.817 DEBUG:teuthology.orchestra.console.smithi046:check power output: Chassis Power is off 2023-12-16T21:44:35.848 DEBUG:teuthology.orchestra.console.smithi161:check power output: Chassis Power is off 2023-12-16T21:44:35.919 INFO:teuthology.orchestra.console.smithi046:Power off completed 2023-12-16T21:44:35.949 INFO:teuthology.orchestra.console.smithi161:Power off completed 2023-12-16T21:44:36.054 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} duration: 806.8901951313019 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=0b95a3a1adfe46e488c13d683036431b status: fail success: false 2023-12-16T21:44:36.055 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-16T21:44:36.145 INFO:teuthology.run:FAIL